The impact of using a validated delirium screening tool and different levels of education on surgical-trauma intensive care unit (STICU) nurses’ knowledge about delirium is unclear.
To measure the impact of using the Intensive Care Delirium Screening Checklist (ICDSC), with or without a multi-faceted education program, on STICU nurses’ knowledge and perceptions of delirium and their ability to evaluate it correctly.
The knowledge and perceptions of subject nurses about delirium, and agreement between the independent assessments of delirium by the subject nurse and by a validated judge (who always used the ICDSC), were compared across 3 phases. Phase 1: No delirium screening tool and no education. Phase 2: ICDSC and minimal education (ie, ICDSC validation study only). Phase 3: ICDSC and multifaceted education (ie, pharmacist-led didactic lecture, Web-based module, and nurse-led bedside training).
Nurses’ knowledge (mean [SD] score out of 10 points) was similar (P = .08) in phase 1 (6.1 [1.4]) and phase 2 (6.5 [1.4]) but was greater (P = .001) in phase 3 (8.2 [1.4]). Agreement between nurses and the validated judge in the assessment of delirium increased from phase 1 (κ = 0.40) to phase 2 (κ = 0.62) to phase 3 (κ = 0.74). Nurses perceived use of the ICDSC as improving their ability to recognize delirium.
Use of a multifaceted education program improves both nurses’ knowledge about delirium and their perceptions about its recognition. Implementation of the ICDSC improves the ability of STICU nurses to evaluate delirium correctly.
Delirium, characterized by inattention, disorganized thinking, and a fluctuating mental status, occurs in up to 70% of patients undergoing mechanical ventilation in surgical-trauma intensive care units (STICUs).1–4 Given that delirium is associated with increased mortality, a longer duration of mechanical ventilation, and the potential for serious sequelae after leaving the ICU (eg, dementia and prolonged neuropsychological impairment), current practice guidelines recommend that patients be routinely screened by using a validated delirium screening tool such as the Intensive Care Delirium Screening Checklist (ICDSC).5–16 However, despite an increasing awareness among ICU clinicians regarding the sequelae of delirium and the ever-increasing use of protocols for delirium screening efforts in ICUs, most patients admitted to ICUs are not routinely screened for delirium.17–23 Common barriers to delirium screening with a validated tool reported by clinicians include the perceptions that the tool takes too long to complete, that use of a tool does not enhance clinicians’ ability to recognize delirium, and that assessment tools are too complex to use.22–24
Although education is increasingly being recognized as a critical component of efforts to implement delirium screening, the types of education that are optimal, the intensity of the pedagogical strategies that should be used, and the members of the ICU team best suited to deliver this education remain unclear.24–27 Pharmacists play a key role in optimizing the care of patients in the ICU and may be well suited to spearhead ICU delirium detection and education efforts, given pharmacists’ daily presence in the ICU, their experience in optimizing sedation therapy, and their involvement with delirium recognition and prevention efforts.8,17,23,28
Although a number of reports describe delirium screening efforts in medical or mixed medical-surgical ICUs, few studies have described the implementation of delirium screening efforts in STICUs.17,18,24–27,29,30 Furthermore, although results of studies on the impact of using a validated delirium screening tool, accompanied by education surrounding its use, on the ability of ICU nurses to recognize delirium have been published, the relative impact of each of these interventions on the knowledge of nurses about delirium and their ability to evaluate delirium remains unknown. We therefore sought to measure the impact of using the ICDSC, with or without a multifaceted education program led by a pharmacist and a nurse, on the knowledge and perceptions of STICU nurses about delirium and their ability to evaluate delirium correctly.
Delirium occurs in up to 70% of mechanically ventilated surgical-trauma intensive care units patients.
A multiprofessional task force was convened to improve delirium detection and management in the ICUs at Carolinas Medical Center, an 813-bed community teaching hospital with 140 adult ICU beds located in Charlotte, North Carolina. As part of this process, the ICDSC was chosen as the delirium screening tool because of its ease of use, its ability to be used in patients with an impaired ability to communicate, and its ability to identify subsyndromal delirium.31–33 Since the Richmond Agitation Sedation Scale (RASS) was already in use at this institution, the first ICDSC domain was adapted to include a RASS score for each descriptor of level of consciousness (Appendix 1).34 The ICDSC worksheet was also modified so that each ICDSC domain contained both expanded descriptions and key questions that could be used by clinicians to determine whether an abnormality in a particular domain was present.27 The ICDSC provides a score from 0 to 8, with a score of 4 or greater being diagnostic for the presence of delirium. Patients with a RASS of −3, −4, or −5 cannot be evaluated for the presence of delirium given their unarousable state.
The 29-bed STICU at Carolinas Medical Center, a level I trauma center with trauma patients making up approximately 25% of the STICU population, was chosen as the first ICU where the ICDSC would be implemented. To this end, a multidisciplinary team consisting of a critical care pharmacist (doctor of pharmacy degree and postgraduate year 2 critical care pharmacy residency training), a postgraduate year 1 pharmacy resident, a clinical nurse (baccalaureate degree and CCRN), and an attending intensivist (board certified in surgical critical care) were charged with developing a process for ICDSC implementation and education. As the initial component of this process, a 3-phase implementation and evaluation study was designed to measure the impact of using the ICDSC in practice and evaluate the impact of using different types and intensities of education during the implementation process (Figure 1). The study was conducted during a 5-month period starting February 1, 2010, and was approved by the institutional review board at Carolinas Medical Center. Twenty nurses with at least 1 year of experience in STICU were recruited and provided consent for participation in the study. Informed consent was not required from patients because their participation was part of an institutional quality improvement effort.
The primary outcome evaluated across the 3 study phases was the effect of each intervention on the knowledge of each bedside nurse about delirium and its detection by using the ICDSC. To this end, a bank of 30 multiple-choice questions was compiled by 3 of the investigators (G.G., A.P.L., J.W.D.). Three different assessments (A, B, and C), each with 10 questions, were randomly pulled from this 30-question bank (Appendix 2). During each phase of the study, subject nurses were randomly assigned to complete 1 of the 3 assessments. No subject nurse received the same assessment twice.
The impact of 2 different educational interventions was evaluated as part of the study (Figure 1). Given that different clinicians retain information by different learning styles, the educational intervention was specifically developed to accommodate kinesthetic, auditory, and visual learners.35 In the phase 2 educational intervention, geared toward visual learners, nurses were provided with a copy of the original 2001 ICDSC validation study to read at least 24 hours before the start of the phase 2 assessment(s) of patients.15 Although no additional education was provided to the subject nurse at this time, the nurse was allowed to ask questions pertaining to use of the ICDSC, although these questions were not encouraged. In the phase 3 educational intervention that incorporated all 3 learning strategies (ie, visual, auditory, and kinesthetic), each nurse subject underwent a multifaceted educational program led by both a pharmacist and the validated judge that consisted of (1) a 30-slide live presentation by a critical care pharmacist, (2) a Webcast education module of the same presentation (Appendix 3), and (3) a bedside demonstration of delirium screening by a validated judge using the ICDSC in at least 1 patient who was not a study subject. The presentation highlighted the consequences of delirium, risk factors for its development, and challenges associated with its detection. Additionally, it included a comprehensive review of how to use the IDCSC properly that was adapted from a previous ICDSC implementation and education effort at Tufts Medical Center.26 Subject nurses were encouraged to ask questions during all components of the intensive education in phase 3.
In addition to measuring the impact of using the ICDSC and the 2 different educational interventions supporting its use on the knowledge of each bedside nurse, we also sought to measure the effect of each of these interventions on the ability of the subject nurses to use the IDCSC correctly. This effect was measured across the 3 study phases by assessing the agreement between the validated judge and the subject nurse for the correct evaluation of delirium (ie, delirium present, delirium absent, or unable to assess patient) for the patient(s) the subject nurse was caring for on the shift where a delirium evaluation was scheduled to take place.
The educational intervention accommodated kinesthetic, auditory, and visual learners.
Subject nurses are assigned to care for up to 2 patients per shift. Patients were excluded if they were less than 18 years of age. Given the importance of allowing the results of our study to be generalizable to all patients admitted to a STICU setting, patients with factors that might have precluded the evaluation of delirium (eg, a patient in a coma related to a traumatic brain injury) were not excluded from the study. The validated judge (B.B.R.), an experienced STICU clinical nurse, served as the gold standard for delirium assessments. She was formally trained regarding use of the ICDSC by an expert in the field of delirium (J.W.D.) at Tufts Medical Center in Boston, Massachusetts, using a process that was used in a prior study26 of implementation of the ICDSC to screen for delirium. Neither routine patient care nor the administration of analgesia, sedation, or psychoactive medications were modified for the purposes of the study; however, changes may have occurred as a component of usual care.
Evaluations included the nurses’ ability to use a delirium tool correctly.
During each phase, the subject nurse was asked if his or her patient had delirium. Subject nurses did not use a delirium screening tool in phase 1; however, they were required to use the ICDSC in phases 2 and 3 (Figure 1). The validated judge used the ICDSC for all delirium assessments. Delirium assessments of the identified patient were conducted first by the subject nurse and then independently, but consecutively, in an immediate fashion, by the validated judge. During each delirium assessment, the validated judge was allowed to question the subject nurse about events that had occurred in the prior 12-hour shift that could be pertinent to the ICDSC evaluation (eg, the number of hours the patient had slept). The results of the validated judge’s assessments were not shared with the subject nurses. Finally, in an effort to characterize nurses’ perceptions about ICU delirium identification and use of the ICDSC in each phase of the study, each subject completed a survey that was adapted from a previously validated survey developed by experts in the field of ICU delirium.22
The proportion of nurses who perceived delirium to be challenging to assess decreased.
Data were presented by using the most appropriate descriptive statistic (eg, means and standard deviations, counts, and percentages). For the primary outcome of nursing knowledge assessed via the multiple-choice test scores (percentage correct), 20 subjects were needed to detect an effect size of 0.7 with an alpha of .05 and a power of 80%. An effect size for a paired Student t test in this situation is the ratio of the mean difference between time periods relative to the standard deviation of that difference. In this case, an effect size of 0.7 means that the underlying difference in means is equal to 0.7 standard deviations. The mean percentages correct from phase 1 versus 2, phase 1 versus 3, and phase 2 versus 3 were compared by using paired Student t tests. A 2-tailed P value less than .05 was considered statistically significant. The κ statistic was used to evaluate agreement between the subject nurses and the validated judge for the correct evaluation of delirium and accounts for agreement beyond that expected by chance. This was evaluated for a quantitative level of significance per the guidelines proposed by Landis and Koch.36 Responses for each survey question about nurses’ perceptions were based on a 5-point Likert scale (1 = strongly disagree, 2 = disagree, 3 = neither agree nor disagree, 4 = agree, 5 = strongly agree). The mean difference between phases 1 and 3 was calculated for each question and evaluated for significance by using a Wilcoxon rank test. SAS, version 9.2, was used for all analyses (SAS Institute, Cary, North Carolina).
A total of 20 nurses were recruited to participate. One subject was lost to follow-up before participation in the first phase of the study. Subject nurses were a mean of 33.8 (SD, 8.7) years old, worked primarily on a day shift (63%), and had worked in critical care, and specifically the STICU at Carolinas Medical Center for a mean of 6.7 (SD, 4.4) and 4.4 (SD, 3) years, respectively. Sixty-three percent had a baccalaureate degree and 53% had CCRN certification. A total of 90 paired assessments were completed during the course of the study in 73 patients, with 32 completed during phase 1, 32 during phase 2, and 26 during phase 3. The mean (SD) and median (interquartile range) number of assessments performed by each subject nurse were 4.74 (0.81) and 5.0 (4.0–5.0), respectively. The patients were predominately male (63%), were a mean age of 55 (SD, 18) years, and had a mean score of 16.5 (SD, 7.7) on the Acute Physiology and Chronic Health Evaluation II at admission. Slightly more than half (55%) were undergoing mechanical ventilation at the time of the delirium assessment and were primarily admitted to either the trauma (39%), general surgical (25%), or transplant (14%) services. Of the trauma patients evaluated, 30% had a diagnosis of traumatic brain injury.
Nursing knowledge was similar between phase 1 (mean, 6.1; SD, 1.4) and phase 2 (mean, 6.5; SD, 1.4; P = .08), but greater in phase 3 (mean, 8.2; SD, 1.4; P = .001). Agreement between subject nurses and the validated judge for the correct evaluation of delirium across the 3 study phases is summarized in the Table. Using the Landis and Koch criteria, agreement improved from fair in phase 1 (κ=0.40; 95% CI, 0.11–0.69) to substantial agreement in phases 2 (κ = 0.62; 95% CI, 0.39–0.69) and remained at this level in phase 3 (κ = 0.74; 95% CI, 0.69–0.95).
Across the 3 study phases, the proportion of nurses who perceived delirium to be challenging to assess in ICU patients decreased (89.5% in phase 1, to 78.9% in phase 2, to 63.2% in phase 3), and the proportion of nurses who agreed with the statement that the ICDSC makes delirium easier to identify in their patient(s) increased (57.9% in phase 1, to 78.9% in phase 2, to 89.5% in phase 3; Figure 2). The mean difference on the 5-point Likert scale between phases 1 and 3 for each of these questions was −0.53 (P = .07) and 0.58 (P = .06), respectively.
Our study is the first to demonstrate in an STICU that the addition of a multifaceted educational intervention that incorporates both didactic and bedside pedagogical strategies and that is delivered by both critical care pharmacists and nurses improves nurses’ knowledge about delirium beyond the knowledge seen with minimal education alone. Furthermore, the study demonstrates that use of the ICDSC improves the ability of nurses to evaluate patients correctly for delirium. The interventions used in our study also improve nurses’ perceptions about the importance of using a validated instrument to screen for delirium. Finally, our study demonstrates that critical care pharmacists can play a key role in collaborating with critical care nurses and physicians when delirium screening and educational efforts are being developed and implemented in the ICU.
Previously described delirium screening efforts in the STICU have been focused on characterizing the reliability between the bedside nurse and a trained expert for assessing the presence of delirium after implementation of the Confusion Assessment Method for the Intensive Care Unit (CAM-ICU) accompanied by a comprehensive education program.17,25,30,37,38 However, by combining these components as 1 intervention, the relative impact of implementing a delirium screening tool or various intensities of education about its use on the ability of nurses to evaluate delirium correctly remains unknown. Because the symptoms of delirium that the CAM-ICU and the ICDSC evaluate differ, one cannot extrapolate results from studies using the CAM-ICU to results of studies using the ICDSC.39,40
A variety of educational strategies have been described to boost efforts to implement delirium screening.17,18,24–27,29,30 These strategies can be categorized as being live or Web-based, formal or informal, scenario-based or not scenario-based, classroom (eg, didactic) or bedside-based, and intermittent or continuous. Presentation content is commonly described to include in-depth descriptions of sedation and delirium scoring tools.17,26,27,29 A simple didactic and bedside educational intervention that incorporated script concordance theory, and that took an average of 1.5 hours to conduct, increased the ability of ICU nurses to evaluate delirium properly by using a scale (12% vs 82%, P < .001) and use this scale correctly (8% vs 62%, P < .001).26 An abbreviated educational intervention that took an average of 15 minutes to complete that accompanied implementation of the ICDSC to physicians in the medical ICU improved their ability to identify delirium correctly.27 Finally, use of printed materials such as handouts, pocket cards, bulletin boards, and posters has been incorporated as a component of the education process.17,18,24,25,29
Use of the Intensive Care Delirium Screening Checklist improved nurses’ ability to evaluate delirium correctly.
Although a plethora of different strategies have been reported to educate ICU clinicians about delirium, few studies17,18,24–27,29,30 have characterized the optimal type and intensity of education that should be used. These data are important because many of the reported barriers to delirium assessment in the ICU are tied to the need for greater education.22–24 However, with an ever-increasing scarcity of health care resources, evidence to justify expanded educational efforts to support ICU delirium screening programs is greatly needed. We found that an intensive strategy consisting of live pharmacist-led lectures, Web-based training, and a nurse-led bedside demonstration improved delirium-related knowledge among nurses far more than simply handing nurses the original ICDSC validation study. In our study, the time required to deliver the multifaceted education to each subject nurse was approximately 45 to 60 minutes. Although our results suggest that institutions will need to devote substantial resources (particularly the time spent by clinical nurse educators and/or pharmacists) to optimize the delivery of a multifaceted education intervention such as the one used in our study, cheaper, more efficient educational strategies may work just as well (eg, a “train-the-trainer” approach, where a trained staff nurse trains their colleagues at the bedside after they have completed the Web-based education).
Reported barriers to delirium assessment in the intensive care unit are tied to the need for greater education.
Perceptions among nurses and pharmacists about delirium and its identification pose a significant barrier to the success of efforts to implement delirium screening.22–24 Educational efforts designed with these barriers in mind may impart a positive influence on such perceptions and contribute to the success of screening. Our educational program included information on not just the epidemiology and consequences of delirium (eg, the importance of screening) but also on how the ICDSC should be properly used. Although the number of nurses who reported that delirium was challenging to assess decreased after education, 63% continued to report that delirium remained challenging to assess. Additionally, although improvements in nurses’ perceptions were noted between phases 1 and 3, the difference between the phases was not significant. These findings highlight the importance of developing a continuous quality improvement plan for delirium screening that measures both screening compliance and quality. Strategies to obtain end-user feedback and provide supplemental didactic and bedside nursing education when needed have been described and should be considered to help sustain the benefits gained during the initial implementation phase.17,25,29
Our study has several strengths, including the fact that we separated implementation of the ICDSC and use of the educational interventions into different phases and that the comprehensive educational intervention evaluated was validated in a prior study.26 Furthermore, the expert evaluator used the ICDSC during all assessments and was formally trained regarding its use at Tufts Medical Center. Last, the external validity of our study is high given that no patients were excluded. The low rate of delirium detected in our study (13%, 12 of 90 assessments) deserves mention because it is substantially lower than the rate detected in other STICU studies.2,3,4,8,18 This can be explained, in part, by our use of a screening tool (ie, the ICDSC) that has a lower reported sensitivity than the tool used in other published studies (ie, CAM-ICU) and our inclusion of patients with conditions where delirium cannot be evaluated (eg, severe traumatic brain injury).39 Additionally, we did not have a requirement for mechanical ventilation and thus, unlike other studies in this population, our patients may have had a lower baseline risk for delirium. Finally, unlike other studies where delirium assessment was performed from ICU admission to the time of discharge, we evaluated patients only at a single point in time. Given the fluctuation of delirium symptoms, increasing the frequency and/or duration of delirium assessments would most likely increase the likelihood for detecting delirium. As most of our patients either did not have delirium or had factors that precluded its assessment, we cannot draw conclusions specific to the subgroup of patients with delirium. Therefore, further investigation is needed in a study that is larger and less inclusive than ours.
Although all paired assessments were conducted as soon as possible after each educational phase was completed, we did not standardize the time between education delivery and the time of the assessments and thus subject nurse recall of the information provided could have varied. It remains to be determined if individual learning styles influence the knowledge gained from multifaceted educational interventions focused on improving use of delirium screening tools, given that we did not evaluate the preferred learning styles of the nurses at baseline. In an effort to enroll nurses who were motivated to complete all phases of the study, we did not randomly select nurses to participate from the study ICU and thus our population of nurses may not have been representative of the entire study STICU. We failed to evaluate the number of nurses who actually read the ICDSC-validating article, and in some situations, nurses may have been forced to complete the multiple choice assessments and surveys in the sometimes loud and distracting environment of the ICU. Finally, we did not collect specific data regarding how long it took each nurse to complete each ICDSC assessment.
Delirium screening with a validated tool is a key component of patient care in the ICU. It is encouraging that subject nurses viewed the ICDSC as an easy-to-use delirium screening method. Delirium screening efforts should be accompanied by an educational intervention that incorporates both didactic and bedside pedagogical methods, uses both live and Web-based strategies, and includes critical care pharmacists in key roles. Implementation of the ICDSC in a STICU improved the ability of nurses to evaluate patients for delirium correctly. Further research is required to investigate the sustainability of the gains observed in our study, the effect of delirium assessments conducted by STICU nurses on patients’ outcomes, and the impact of implementing the ICDSC and multifaceted education about its use in other types of ICUs.
We thank the nurses of the STICU for making this study possible, Meredith Moore, rn, for her contributions to the data analysis, and Yoanna Skrobik, md, for her review of the manuscript.
Financial support for this project was provided by the Departments of Nursing and Pharmacy at the Carolinas Medical Center in Charlotte, North Carolina.
Now that you’ve read the article, create or contribute to an online discussion on this topic. Visit www.ajcconline.org and click “Submit a response” in either the full-text or PDF view of the article.
To purchase electronic or print reprints, contact The InnoVision Group, 101 Columbia, Aliso Viejo, CA 92656. Phone, (800) 899-1712 or (949) 362-2050 (ext 532); fax, (949) 362-2049; e-mail, firstname.lastname@example.org.