Feasibility of Nurse-Led Multidimensional Outcome Assessments in the Neuroscience Intensive Care Unit

This quality improvement project was designed to improve understanding of the postdischarge outcomes of critically ill patients admitted to the NICU at the University of Cincinnati Medical Center. This 771-bed academic hospital system serves the surrounding community and acts as a tertiary care facility for a catchment area of approximately 2.5 million people living in the Greater Cincinnati and Northern Kentucky regions. The NICU is a 20-bed unit and is the region’s only dedicated unit staffed by board-certified neurointensivists and specialty-trained critical care nurses. Although quality improvement efforts did not require specific institutional review board approval, our retrospective reporting of these data were approved by the institutional review board of the University of Cincinnati with a waiver of informed consent.

We began our quality improvement project by identifying all patients who were admitted to the NICU between November 2 and November 16, 2015 (pilot phase). We subsequently enrolled consecutive patients admitted from January 1, 2016, to July 1, 2017 (implementation phase; Figure 1). For the first 90 days of the quality improvement project, NICU patients were identified on the basis of clinical census data by investigators (B.F., E.S.) while an electronic health record–based query was simultaneously developed and verified against clinical census data.

During the implementation phase, data were entered using REDCap (Research Electronic Data Capture), a secure, web-based application.¹²  Demographic information was collected including age, race, primary diagnosis, and disposition at discharge. We included all patients with more than 24 hours of critical care provided by a neurointensivist and all those with in-hospital mortality regardless of length of stay. We excluded patients admitted during the immediate postoperative period or for whom intensive care was provided for 24 hours or less, those cared for by a nonneurocritical care service (eg, pulmonary critical care), and those readmitted within 6 months of a previous neurocritical care diagnosis. Inclusion and primary diagnosis data were adjudicated at a weekly conference attended by an NICU nurse and at least 1 NICU physician.

Once neurocritically ill survivors were identified and verified, we performed a scripted telephone assessment between 3 and 6 months after admission. Scripted, structured interviews were coded using branching logic within REDCap. All calls were performed by either a clinical nurse, a physician, or a dedicated research coordinator. All personnel were trained by one of the authors (E.S. or O.L.), and their ability to perform the scripted telephone assessment was verified under direct observation. Patients were called at least twice, and if there was no answer or if a callback was requested, additional calls could be made at the discretion of the caller. For the pilot phase, the duration of each phone call was recorded to determine the feasibility of implementation.

Telephone assessments included a standardized, performance-based cognitive assessment using the modified Telephone Interview for Cognitive Status (mTICS),¹³  along with patient-reported assessments of functional outcome using the modified Rankin Scale (mRS)¹⁴  and the Glasgow Outcome Scale-Extended (GOSE).^15,16  The patient-reported 5-level EQ-5D (EQ-5D-5L), developed by the EuroQol Group, was used for quality-of-life assessment, along with an estimate of a “visual analog scale,” ranking health on a scale of 0 to 100.¹⁷ 

Data were analyzed using R (version 3.4.3) and are reported as mean (SD) or median (interquartile range [IQR]), as appropriate. To identify patient-level factors that were associated with response to follow-up phone call, we compared survivors who answered versus those who did not answer using the χ² test, Wilcoxon rank sum test, or t test, as appropriate. A response was considered if at least 1 measure was completed. Statistical significance was set at P less than .05 by convention. A logistic regression model was created to estimate the probability of a call being answered with repeated attempts to extrapolate the need for additional calls.

During the pilot phase, 49 patients were identified over a 2-week period. The mean (SD) age was 50 (17.5) years, and 31 of the patients (63%) were men. Primary diagnoses included ischemic stroke (15 patients; 31%), seizures or status epilepticus (10 patients; 20%), TBI (5 patients; 10%), intracerebral hemorrhage (3 patients; 6%), and SAH (2 patients; 4%). Of these patients, 8 (16%) died in the hospital and another 4 (8%) died after discharge, for a 3-month mortality rate of 24%. A total of 10 patients (20%) were lost to follow-up. Data entry for those who died or were lost to follow-up took study personnel 3.1 minutes per patient. Completed telephone assessments required 21.9 minutes per patient. For an average 2-week census, we estimated 11 hours of personnel time (Table 1).

During the implementation phase, we identified 2163 patients over an 18-month period. Of those, 106 were admitted to the NICU but cared for by a nonneurocriti-cal care team, 665 were admitted for 24 hours or less or during the immediate postoperative period alone, and 68 were readmitted for a recent or ongoing neurocritical care illness. A total of 1324 of 2163 patients fulfilled our inclusion criteria (Table 2). A total of 365 of 1324 (27.6%) died in the hospital and another 150 (11.3%) died after discharge, for a follow-up mortality of 515 of 1324 (38.9%). Of those who died, 385 (74.8%) died as a result of withdrawal of care decisions.

A total of 959 hospital survivors were slated for follow-up; 17 were missed and did not receive a call, and 90 were found to have died on medical record review. A telephone call was made to the remainder at a mean (SD) time of 4.4 (0.8) months after admission; 596 patients or their caregivers answered telephone calls, whereas 253 did not. Three had incorrect contact information. We found that neither age, sex, race, primary diagnosis, hospital disposition, nor time of day was significantly associated with a lack of answer, whereas the length of time from admission to attempted follow-up was longer for those who never answered telephone calls after repeated attempts (Table 3). The probability of a survivor or caregiver answering a call decreased by 31% for each additional attempt (odds ratio, 0.69; 95% CI, 0.57-0.83; Figure 2). Among the survivors or caregivers who answered calls, 94% of them did so by the second attempt to contact them.

Of the 596 answered calls, 60 respondents reported that their family member had died and 32 declined to participate; 5 were nonprimary caregivers, whereas the rest identified as patients or primary caregivers. A further 8 participated in only a portion of the call without completing any measurement. The resulting total loss to follow-up was 313 of 1324 patients (23.6%). The majority of assessments (261 of 496; 52.6%) were made by a single caller (O.L.).

We found that 377 of 496 (76.0%) patients were at home at the time of follow-up, whereas 97 (19.6%) remained in short- or long-term care facilities. Excluding those who died, 285 of 496 (57.5%) had a poor outcome defined as a GOSE score of < 5; the median (IQR) GOSE score was 4 (3-6). A total of 272 of 495 patients (54.9%) had poor outcome defined as an mRS score of over 2; the median (IQR) mRS score was 3 (2-4). A total of 223 of 496 patients (45.0%) participated in cognitive testing with a mean (SD) score of 34.9 (5.3), and a total of 45 of 223 (20.2%) had scores below the optimal cutoff for cognitive impairment.¹³  Of the 474 of 496 (95.6%) who participated at least in part in the EQ-5D-5L, patients or their caregivers reported moderate, severe, or extreme problems with mobility (224 of 474; 47.3%), performing usual activities (207 of 474; 43.7%), performing self-care (181 of 474; 38.2%), anxiety or depression (172 of 474; 36.3%), and pain (163 of 473; 34.5%). The overall quality of life, measured on a scale from 0 to 100, averaged 60.8 (25.2).

We identified critically ill patients admitted to the NICU in a quality improvement effort designed to increase our understanding of patient outcomes after discharge in a neurocritical care patient population. We found that a standardized, multidimensional outcome assessment by telephone was feasible and required approximately 11 hours of total time for an average 2-week census. We also found that limiting attempts to obtain follow-up information via phone to 2 calls per patient was acceptable and efficient. We had a loss to follow-up rate of 23.6% across a broad range of neurocritical care disease states at a mean (SD) time of 4.4 (0.8) months after admission. Our findings suggest that it is feasible to collect important patient-reported outcome measures via periodic telephone follow-ups in a diverse neurocritical care patient population.

Our cohort was broadly representative of the neurocritical care population. In 4 cohort studies encompassing 4844 patients across the United States and Europe,^8-11  patients admitted to the NICU included 15% to 34% with ischemic stroke, 9% to 27% with intracerebral hemorrhage, and 5% to 19% with epilepsy including status epilepticus. Three of these studies included patients with SAH, making up 5% to 16% of the population.^8,9,11  Our cohort included nearly identical proportions of patients with each diagnosis despite the fact that the European studies specifically excluded primarily neurosurgical patients. At our center, both neurology and neurosurgical patients requiring intensive care were admitted to the same NICU. Only 2 studies included patients with TBI, who made up 3% of patients in the US cohort,¹¹  compared with 19% in Europe⁸  and 17% in our study.

In 2381 patients admitted to a US NICU over a 39-month period,¹¹  mortality at 1 year was found to be 25.4%, compared with 38.9% at a mean (SD) of 4.4 (0.8) months in our study. Among patients who died in the previous study, 61.1% underwent withdrawal of life-sustaining therapy, as compared with nearly three-quarters in our study. In 796 patients from Germany,⁹  the 1-year mortality rate was 38.1% despite the exclusion of patients with early do-not-treat orders and those with TBI. The rate of withdrawal of life-sustaining therapy was only 5%, but this cohort included only patients with specific critical care needs, such as mechanical ventilation, suggesting higher acuity. Finally, in 1155 patients admitted to an Austrian NICU,⁸  mortality at a mean (SD) time of 2.7 (1.0) years was 35.2%. This population was similar in terms of diagnoses to our cohort, but withdrawal of life-sustaining therapy decisions were not documented. Together, these studies suggest variability in mortality related to both disease severity and subsequent treatment decisions.

According to a scoping review of the general critical care literature, less than 2% of studies included assessment of postdischarge outcome.³  Of those cohort studies that capture postdischarge outcome, the median sample size was 107 patients assessed at a median (IQR) of 6 months (6-12 months) after discharge.³  We assessed postdischarge outcomes in 1324 patients at a mean (SD) of 4.4 (0.8) months after admission. The median (IQR) follow-up rate across the general critical care literature was 80% (65%-93%), which is comparable to our rate of 76.4%. Specifically for cohort studies with 1 follow-up assessment, a median (IQR) of 22% (7%-37%) of patients were reported lost to follow-up, similar to our rate of 23.6%.³  In the neurocritically ill population, the proportion of patients lost to follow-up ranges broadly. In the United States, mortality data at 1 year were available for 99.2% of patients¹¹  using a combination of medical record review and social security data. Using the mRS at 1 year, a European study was able to follow 96.4% of patients using a mailed questionnaire followed by a physician telephone call if the mailer was not returned.⁹  In the study closest to ours in terms of population, follow-up using a combination of the Glasgow Outcome Scale (GOS) and mRS via telephone was obtained in 75.8% of patients at a mean (SD) of 2.7 (1.0) years.⁸  Although methods exist for extracting pure mortality data with low loss to follow-up, capturing multidimensional outcome assessments at a single postdischarge time point may require a balance between available resources and rates of missing data.

Overall, 45% of the general critical care literature includes more than 1 domain in its outcome assessment, and 14% measure more than 2 domains.³  We measured outcomes across 3 domains: cognitive activity limitations with the mTICS, participation restriction with the GOSE and the mRS, and quality of life with the multidimensional EQ-5D-5L. Our assessments of physical and mental health impairment were patient or caregiver reported rather than direct measurements, although performance-based testing would not be practical across such large numbers of patients, many of whom are unable to travel to a central site for a quality improvement initiative. We found that more than three-quarters of patients were at home at follow-up, yet nearly half described moderate, severe, or extreme problems with mobility; more than one-third had moderate, severe, or extreme problems with anxiety, depression, or pain; and 20% who were able to participate in cognitive testing had scores consistent with mild cognitive impairment. Nonetheless, mean (SD) quality of life as measured on a continuous scale from 0 to 100 was 60.8 (25.2), somewhere between the mean for healthy individuals in the United Kingdom (77.8 [18.6])¹⁷  and those with stroke (49.5 [26.2]).¹⁸  These findings are novel and may serve as a benchmark for expected outcomes across the spectrum of survivors of neurocritical care.

We specifically chose telephone measurements with high validity within the neurocritical care population. The GOS and GOSE have been widely used as a primary outcome measure across studies of patients with TBI. In recent work using the Transforming Research and Clinical Knowledge in TBI Pilot data at 6 months after injury, the GOSE modestly correlated with cognitive and emotional health, indicating a complex interplay among multiple domains, thereby limiting its use as a single measure.¹⁶  After SAH, prediction models of 1-year outcome have been developed across multiple dimensions using the mRS, mTICS, and Sickness Impact Profile as a measure of quality of life; overlap was observed in the clinical variables associated with prognosis for each dimension.⁵  After stroke, the mRS has been correlated with the EQ-5D, suggesting a similar interplay between functioning and quality-of-life domains¹⁸ ; the combination of the 2 measures has been used in part to generate utility-weighted versions of the mRS.¹⁹  The challenge in the neurocritical care population is that neurological injury often impairs motor or cognitive functioning independently of critical illness, and the interplay between neurological injury and critical illness remains to be disentangled.

Our study was limited in that it was a quality improvement initiative rather than a formal clinical observational cohort study. We did not use trained neuropsychologists but relied on standardized REDCap structured interviews to guide calls using validated instruments. Although one of the authors performed the majority of the data entry and quality assurance (E.S.) and another the majority of the phone call follow-up (O.L.), different personnel participated over time, which may have altered the efficiency of the initiative. We did not use double data entry or establish interrater agreement; however, all callers were clinical nurses or physicians with experience in neurocritical care and data were reviewed for consistency by study authors (B.F., S.F.). Finally, we chose telephone contact in part because of concern that the population in our catchment area might not have reliable home addresses or easy access to internet resources and/or might not necessarily follow up in person at our hospital. This concern precluded the use of newer web-based testing methods such as PROMIS (Patient-Reported Outcomes Measurement Information System; http://www.healthmeasures.net/explore-measurement-systems/promis), which may offer more comprehensive and efficient methods of obtaining patient-reported outcomes.

We found that a nursing-led multidimensional telephone-based outcome initiative was feasible for patients with neurocritical illness after discharge. We found the data locally useful in understanding the spectrum of outcomes for patients. We were able to disseminate these data within our multidisciplinary morbidity and mortality conferences, and a better understanding of outcomes resulting from this work informs health care discussions with both patients and their caregivers. Our cohort was representative of neurocritical care populations internationally, and outcome measures should be generalizable as a result. More research is needed to develop a comprehensive understanding of domain-specific impairments related to specific neurocritical illnesses. A more nuanced definition of post–intensive care syndrome in the neurocritical care population requires that the effects of critical illness be recognized as independent from the functional and/ or cognitive deficits incurred by a primary neurological injury.

The authors acknowledge the support of the UCHealth neuroscience nursing management and the support and dedication of the NICU nurses during the course of the project. They also wish to thank Mitchell Singstock and Cody Cooper, whose efforts on this improvement project were invaluable. Finally, a special thank you is due to Karen Burk for her role as a member of the outcomes team in which she participated in adjudicating cases alongside the authors. Her expertise in neurocritical care case reporting and follow-up through her years of dedicated service to the Division was invaluable.