Background

The outcome focus for survivors of critical care has shifted from mortality to patient-centered outcomes. Multidimensional outcome assessments performed in critically ill patients typically exclude those with primary neurological injuries.

Objective

To determine the feasibility of measurements of physical function, cognition, and quality of life in patients requiring neurocritical care.

Methods

This evaluation of a quality improvement initiative involved all patients admitted to the neuroscience intensive care unit at the University of Cincinnati Medical Center.

Interventions

Telephone assessments of physical function (Glasgow Outcome Scale-Extended and modified Rankin Scale scores), cognition (modified Telephone Interview for Cognitive Status), and quality of life (5-level EQ-5D) were conducted between 3 and 6 months after admission.

Results

During the 2-week pilot phase, the authors contacted and completed data entry for all patients admitted to the neuroscience intensive care unit over a 2-week period in approximately 11 hours. During the 18-month implementation phase, the authors followed 1324 patients at a mean (SD) time of 4.4 (0.8) months after admission. Mortality at follow-up was 38.9%; 74.8% of these patients underwent withdrawal of care. The overall loss to follow-up rate was 23.6%. Among all patients contacted, 94% were available by the second attempt to interview them by telephone.

Conclusions

Obtaining multidimensional outcome assessments by telephone across a diverse population of neurocritically ill patients was feasible and efficient. The sample was similar to those in other cohort studies in the neurocritical care population, and the loss to follow-up rate was comparable with that of the general critical care population.

Intensive care encompasses a host of lifesaving and life-sustaining services that make survival possible even after the most threatening diseases. Although critical care costs the United States $108 billion each year and accounts for 13.2% of hospital costs,1  survival also incurs considerable costs. In addition to recovering from their critical illness, survivors often experience new or worsening impairments in functional ability, cognition, and/or mental health—a syndrome referred to as post–intensive care syndrome.2  To understand this syndrome, long-term follow-up after discharge is required to assess multiple domains: physical, cognitive, behav-ioral, and quality of life.

National and international critical care societies have recommended prioritizing research on outcomes in critical care survivors rather than on mortality alone. To date, most high-quality outcomes-based research has been performed in the general critical care population.3  Substantial efforts have been made to harmonize metrics in the acute respiratory distress syndrome population specifically,4  with a focus on cognition and quality of life in addition to assessments of physical functioning. However, such high-impact critical care outcomes research has largely excluded neurocritically ill patients as well as patients with preexisting neurological comorbidities. Disease-specific multidimensional outcomes have been assessed in patients with subarachnoid hemorrhage (SAH),5  common data elements have been suggested for traumatic brain injury (TBI),6  and stroke outcomes research has incorporated standardized patient-centered outcomes.7  However, the lack of consensus around these outcome measures and the importance of studying outcomes in different domains has made it difficult to assess comparative effectiveness for interventions in diverse neuro-critically ill patient populations. Cohort studies from neuroscience intensive care units (NICUs) have focused on outcomes in the functional domain alone.8-11 

In a quality improvement initiative, we sought to follow up with patients with critical neurological injuries via telephone to gather data regarding outcomes in the physical functioning, cognition, and quality-of-life domains using a standard battery of validated instruments. Our nursing-led strategy’s goals were to efficiently capture the complexity of the diagnoses and outcomes of patients admitted to the NICU and to obtain valuable insights into the challenges of obtaining data from patients and their families after a neurological injury. We hypothesized that our strategy would be feasible and efficient and would result in an acceptable rate of follow-up.

Methods

This quality improvement project was designed to improve understanding of the postdischarge outcomes of critically ill patients admitted to the NICU at the University of Cincinnati Medical Center. This 771-bed academic hospital system serves the surrounding community and acts as a tertiary care facility for a catchment area of approximately 2.5 million people living in the Greater Cincinnati and Northern Kentucky regions. The NICU is a 20-bed unit and is the region’s only dedicated unit staffed by board-certified neurointensivists and specialty-trained critical care nurses. Although quality improvement efforts did not require specific institutional review board approval, our retrospective reporting of these data were approved by the institutional review board of the University of Cincinnati with a waiver of informed consent.

We began our quality improvement project by identifying all patients who were admitted to the NICU between November 2 and November 16, 2015 (pilot phase). We subsequently enrolled consecutive patients admitted from January 1, 2016, to July 1, 2017 (implementation phase; Figure 1). For the first 90 days of the quality improvement project, NICU patients were identified on the basis of clinical census data by investigators (B.F., E.S.) while an electronic health record–based query was simultaneously developed and verified against clinical census data.

During the implementation phase, data were entered using REDCap (Research Electronic Data Capture), a secure, web-based application.12  Demographic information was collected including age, race, primary diagnosis, and disposition at discharge. We included all patients with more than 24 hours of critical care provided by a neurointensivist and all those with in-hospital mortality regardless of length of stay. We excluded patients admitted during the immediate postoperative period or for whom intensive care was provided for 24 hours or less, those cared for by a nonneurocritical care service (eg, pulmonary critical care), and those readmitted within 6 months of a previous neurocritical care diagnosis. Inclusion and primary diagnosis data were adjudicated at a weekly conference attended by an NICU nurse and at least 1 NICU physician.

Once neurocritically ill survivors were identified and verified, we performed a scripted telephone assessment between 3 and 6 months after admission. Scripted, structured interviews were coded using branching logic within REDCap. All calls were performed by either a clinical nurse, a physician, or a dedicated research coordinator. All personnel were trained by one of the authors (E.S. or O.L.), and their ability to perform the scripted telephone assessment was verified under direct observation. Patients were called at least twice, and if there was no answer or if a callback was requested, additional calls could be made at the discretion of the caller. For the pilot phase, the duration of each phone call was recorded to determine the feasibility of implementation.

This quality improvement project was designed to improve understanding of the postdischarge outcomes of critically ill patients.

Telephone assessments included a standardized, performance-based cognitive assessment using the modified Telephone Interview for Cognitive Status (mTICS),13  along with patient-reported assessments of functional outcome using the modified Rankin Scale (mRS)14  and the Glasgow Outcome Scale-Extended (GOSE).15,16  The patient-reported 5-level EQ-5D (EQ-5D-5L), developed by the EuroQol Group, was used for quality-of-life assessment, along with an estimate of a “visual analog scale,” ranking health on a scale of 0 to 100.17 

Data were analyzed using R (version 3.4.3) and are reported as mean (SD) or median (interquartile range [IQR]), as appropriate. To identify patient-level factors that were associated with response to follow-up phone call, we compared survivors who answered versus those who did not answer using the χ2 test, Wilcoxon rank sum test, or t test, as appropriate. A response was considered if at least 1 measure was completed. Statistical significance was set at P less than .05 by convention. A logistic regression model was created to estimate the probability of a call being answered with repeated attempts to extrapolate the need for additional calls.

Results

Pilot Feasibility

During the pilot phase, 49 patients were identified over a 2-week period. The mean (SD) age was 50 (17.5) years, and 31 of the patients (63%) were men. Primary diagnoses included ischemic stroke (15 patients; 31%), seizures or status epilepticus (10 patients; 20%), TBI (5 patients; 10%), intracerebral hemorrhage (3 patients; 6%), and SAH (2 patients; 4%). Of these patients, 8 (16%) died in the hospital and another 4 (8%) died after discharge, for a 3-month mortality rate of 24%. A total of 10 patients (20%) were lost to follow-up. Data entry for those who died or were lost to follow-up took study personnel 3.1 minutes per patient. Completed telephone assessments required 21.9 minutes per patient. For an average 2-week census, we estimated 11 hours of personnel time (Table 1).

Implementation: Patient Identification

During the implementation phase, we identified 2163 patients over an 18-month period. Of those, 106 were admitted to the NICU but cared for by a nonneurocriti-cal care team, 665 were admitted for 24 hours or less or during the immediate postoperative period alone, and 68 were readmitted for a recent or ongoing neurocritical care illness. A total of 1324 of 2163 patients fulfilled our inclusion criteria (Table 2). A total of 365 of 1324 (27.6%) died in the hospital and another 150 (11.3%) died after discharge, for a follow-up mortality of 515 of 1324 (38.9%). Of those who died, 385 (74.8%) died as a result of withdrawal of care decisions.

Implementation: Patient Contact

A total of 959 hospital survivors were slated for follow-up; 17 were missed and did not receive a call, and 90 were found to have died on medical record review. A telephone call was made to the remainder at a mean (SD) time of 4.4 (0.8) months after admission; 596 patients or their caregivers answered telephone calls, whereas 253 did not. Three had incorrect contact information. We found that neither age, sex, race, primary diagnosis, hospital disposition, nor time of day was significantly associated with a lack of answer, whereas the length of time from admission to attempted follow-up was longer for those who never answered telephone calls after repeated attempts (Table 3). The probability of a survivor or caregiver answering a call decreased by 31% for each additional attempt (odds ratio, 0.69; 95% CI, 0.57-0.83; Figure 2). Among the survivors or caregivers who answered calls, 94% of them did so by the second attempt to contact them.

Implementation: Patient Follow-up

Of the 596 answered calls, 60 respondents reported that their family member had died and 32 declined to participate; 5 were nonprimary caregivers, whereas the rest identified as patients or primary caregivers. A further 8 participated in only a portion of the call without completing any measurement. The resulting total loss to follow-up was 313 of 1324 patients (23.6%). The majority of assessments (261 of 496; 52.6%) were made by a single caller (O.L.).

We found that 377 of 496 (76.0%) patients were at home at the time of follow-up, whereas 97 (19.6%) remained in short- or long-term care facilities. Excluding those who died, 285 of 496 (57.5%) had a poor outcome defined as a GOSE score of < 5; the median (IQR) GOSE score was 4 (3-6). A total of 272 of 495 patients (54.9%) had poor outcome defined as an mRS score of over 2; the median (IQR) mRS score was 3 (2-4). A total of 223 of 496 patients (45.0%) participated in cognitive testing with a mean (SD) score of 34.9 (5.3), and a total of 45 of 223 (20.2%) had scores below the optimal cutoff for cognitive impairment.13  Of the 474 of 496 (95.6%) who participated at least in part in the EQ-5D-5L, patients or their caregivers reported moderate, severe, or extreme problems with mobility (224 of 474; 47.3%), performing usual activities (207 of 474; 43.7%), performing self-care (181 of 474; 38.2%), anxiety or depression (172 of 474; 36.3%), and pain (163 of 473; 34.5%). The overall quality of life, measured on a scale from 0 to 100, averaged 60.8 (25.2).

Discussion

We identified critically ill patients admitted to the NICU in a quality improvement effort designed to increase our understanding of patient outcomes after discharge in a neurocritical care patient population. We found that a standardized, multidimensional outcome assessment by telephone was feasible and required approximately 11 hours of total time for an average 2-week census. We also found that limiting attempts to obtain follow-up information via phone to 2 calls per patient was acceptable and efficient. We had a loss to follow-up rate of 23.6% across a broad range of neurocritical care disease states at a mean (SD) time of 4.4 (0.8) months after admission. Our findings suggest that it is feasible to collect important patient-reported outcome measures via periodic telephone follow-ups in a diverse neurocritical care patient population.

Our cohort was broadly representative of the neurocritical care population. In 4 cohort studies encompassing 4844 patients across the United States and Europe,8-11  patients admitted to the NICU included 15% to 34% with ischemic stroke, 9% to 27% with intracerebral hemorrhage, and 5% to 19% with epilepsy including status epilepticus. Three of these studies included patients with SAH, making up 5% to 16% of the population.8,9,11  Our cohort included nearly identical proportions of patients with each diagnosis despite the fact that the European studies specifically excluded primarily neurosurgical patients. At our center, both neurology and neurosurgical patients requiring intensive care were admitted to the same NICU. Only 2 studies included patients with TBI, who made up 3% of patients in the US cohort,11  compared with 19% in Europe8  and 17% in our study.

In 2381 patients admitted to a US NICU over a 39-month period,11  mortality at 1 year was found to be 25.4%, compared with 38.9% at a mean (SD) of 4.4 (0.8) months in our study. Among patients who died in the previous study, 61.1% underwent withdrawal of life-sustaining therapy, as compared with nearly three-quarters in our study. In 796 patients from Germany,9  the 1-year mortality rate was 38.1% despite the exclusion of patients with early do-not-treat orders and those with TBI. The rate of withdrawal of life-sustaining therapy was only 5%, but this cohort included only patients with specific critical care needs, such as mechanical ventilation, suggesting higher acuity. Finally, in 1155 patients admitted to an Austrian NICU,8  mortality at a mean (SD) time of 2.7 (1.0) years was 35.2%. This population was similar in terms of diagnoses to our cohort, but withdrawal of life-sustaining therapy decisions were not documented. Together, these studies suggest variability in mortality related to both disease severity and subsequent treatment decisions.

According to a scoping review of the general critical care literature, less than 2% of studies included assessment of postdischarge outcome.3  Of those cohort studies that capture postdischarge outcome, the median sample size was 107 patients assessed at a median (IQR) of 6 months (6-12 months) after discharge.3  We assessed postdischarge outcomes in 1324 patients at a mean (SD) of 4.4 (0.8) months after admission. The median (IQR) follow-up rate across the general critical care literature was 80% (65%-93%), which is comparable to our rate of 76.4%. Specifically for cohort studies with 1 follow-up assessment, a median (IQR) of 22% (7%-37%) of patients were reported lost to follow-up, similar to our rate of 23.6%.3  In the neurocritically ill population, the proportion of patients lost to follow-up ranges broadly. In the United States, mortality data at 1 year were available for 99.2% of patients11  using a combination of medical record review and social security data. Using the mRS at 1 year, a European study was able to follow 96.4% of patients using a mailed questionnaire followed by a physician telephone call if the mailer was not returned.9  In the study closest to ours in terms of population, follow-up using a combination of the Glasgow Outcome Scale (GOS) and mRS via telephone was obtained in 75.8% of patients at a mean (SD) of 2.7 (1.0) years.8  Although methods exist for extracting pure mortality data with low loss to follow-up, capturing multidimensional outcome assessments at a single postdischarge time point may require a balance between available resources and rates of missing data.

Overall, 45% of the general critical care literature includes more than 1 domain in its outcome assessment, and 14% measure more than 2 domains.3  We measured outcomes across 3 domains: cognitive activity limitations with the mTICS, participation restriction with the GOSE and the mRS, and quality of life with the multidimensional EQ-5D-5L. Our assessments of physical and mental health impairment were patient or caregiver reported rather than direct measurements, although performance-based testing would not be practical across such large numbers of patients, many of whom are unable to travel to a central site for a quality improvement initiative. We found that more than three-quarters of patients were at home at follow-up, yet nearly half described moderate, severe, or extreme problems with mobility; more than one-third had moderate, severe, or extreme problems with anxiety, depression, or pain; and 20% who were able to participate in cognitive testing had scores consistent with mild cognitive impairment. Nonetheless, mean (SD) quality of life as measured on a continuous scale from 0 to 100 was 60.8 (25.2), somewhere between the mean for healthy individuals in the United Kingdom (77.8 [18.6])17  and those with stroke (49.5 [26.2]).18  These findings are novel and may serve as a benchmark for expected outcomes across the spectrum of survivors of neurocritical care.

We specifically chose telephone measurements with high validity within the neurocritical care population. The GOS and GOSE have been widely used as a primary outcome measure across studies of patients with TBI. In recent work using the Transforming Research and Clinical Knowledge in TBI Pilot data at 6 months after injury, the GOSE modestly correlated with cognitive and emotional health, indicating a complex interplay among multiple domains, thereby limiting its use as a single measure.16  After SAH, prediction models of 1-year outcome have been developed across multiple dimensions using the mRS, mTICS, and Sickness Impact Profile as a measure of quality of life; overlap was observed in the clinical variables associated with prognosis for each dimension.5  After stroke, the mRS has been correlated with the EQ-5D, suggesting a similar interplay between functioning and quality-of-life domains18 ; the combination of the 2 measures has been used in part to generate utility-weighted versions of the mRS.19  The challenge in the neurocritical care population is that neurological injury often impairs motor or cognitive functioning independently of critical illness, and the interplay between neurological injury and critical illness remains to be disentangled.

Patients described problems with mobility and anxiety, depression, or pain; and 20% who were able to participate in cognitive testing had scores consistent with mild cognitive impairment.

Limitations

Our study was limited in that it was a quality improvement initiative rather than a formal clinical observational cohort study. We did not use trained neuropsychologists but relied on standardized REDCap structured interviews to guide calls using validated instruments. Although one of the authors performed the majority of the data entry and quality assurance (E.S.) and another the majority of the phone call follow-up (O.L.), different personnel participated over time, which may have altered the efficiency of the initiative. We did not use double data entry or establish interrater agreement; however, all callers were clinical nurses or physicians with experience in neurocritical care and data were reviewed for consistency by study authors (B.F., S.F.). Finally, we chose telephone contact in part because of concern that the population in our catchment area might not have reliable home addresses or easy access to internet resources and/or might not necessarily follow up in person at our hospital. This concern precluded the use of newer web-based testing methods such as PROMIS (Patient-Reported Outcomes Measurement Information System; http://www.healthmeasures.net/explore-measurement-systems/promis), which may offer more comprehensive and efficient methods of obtaining patient-reported outcomes.

Conclusions

We found that a nursing-led multidimensional telephone-based outcome initiative was feasible for patients with neurocritical illness after discharge. We found the data locally useful in understanding the spectrum of outcomes for patients. We were able to disseminate these data within our multidisciplinary morbidity and mortality conferences, and a better understanding of outcomes resulting from this work informs health care discussions with both patients and their caregivers. Our cohort was representative of neurocritical care populations internationally, and outcome measures should be generalizable as a result. More research is needed to develop a comprehensive understanding of domain-specific impairments related to specific neurocritical illnesses. A more nuanced definition of post–intensive care syndrome in the neurocritical care population requires that the effects of critical illness be recognized as independent from the functional and/ or cognitive deficits incurred by a primary neurological injury.

A more nuanced definition of post–intensive care syndrome requires that the effects of critical illness be recognized as independent from the functional and/or cognitive deficits incurred by a primary neurological injury.

Acknowledgments

The authors acknowledge the support of the UCHealth neuroscience nursing management and the support and dedication of the NICU nurses during the course of the project. They also wish to thank Mitchell Singstock and Cody Cooper, whose efforts on this improvement project were invaluable. Finally, a special thank you is due to Karen Burk for her role as a member of the outcomes team in which she participated in adjudicating cases alongside the authors. Her expertise in neurocritical care case reporting and follow-up through her years of dedicated service to the Division was invaluable.

References

References
1
Halpern
NA
,
Pastores
SM
.
Critical care medicine beds, use, occupancy, and costs in the United States: a methodological review
.
Crit Care Med
.
2015
;
43
(
11
):
2452
-
2459
.
2
Needham
DM
,
Davidson
J
,
Cohen
H
, et al
.
Improving long-term outcomes after discharge from intensive care unit: report from a stakeholders’ conference
.
Crit Care Med
.
2012
;
40
(
2
):
502
-
509
.
3
Turnbull
AE
,
Rabiee
A
,
Davis
WE
, et al
.
Outcome measurement in ICU survivorship research from 1970 to 2013: a scoping review of 425 publications
.
Crit Care Med
.
2016
;
44
(
7
):
1267
-
1277
.
4
Turnbull
AE
,
Sepulveda
KA
,
Dinglas
VD
,
Chessare
CM
,
Bingham
CO
3rd
,
Needham
DM
.
Core domains for clinical research in acute respiratory failure survivors: an international modified Delphi consensus study
.
Crit Care Med
.
2017
;
45
(
6
):
1001
-
1010
.
5
Witsch
J
,
Frey
HP
,
Patel
S
, et al
.
Prognostication of long-term outcomes after subarachnoid hemorrhage: the FRESH score
.
Ann Neurol
.
2016
;
80
(
1
):
46
-
58
.
6
Wilde
EA
,
Whiteneck
GG
,
Bogner
J
, et al
.
Recommendations for the use of common outcome measures in traumatic brain injury research
.
Arch Phys Med Rehabil
.
2010
;
91
(
11
):
1650
-
1660.e17
. doi:
7
Salinas
J
,
Sprinkhuizen
SM
,
Ackerson
T
, et al
.
An international standard set of patient-centered outcome measures after stroke
.
Stroke
.
2016
;
47
(
1
):
180
-
186
.
8
Broessner
G
,
Helbok
R
,
Lackner
P
, et al
.
Survival and long-term functional outcome in 1,155 consecutive neurocritical care patients
.
Crit Care Med
.
2007
;
35
(
9
):
2025
-
2030
.
9
Kiphuth
IC
,
Schellinger
PD
,
Köhrmann
M
, et al
.
Predictors for good functional outcome after neurocritical care
.
Crit Care
.
2010
;
14
(
4
):
R136
. doi:
10
Backhaus
R
,
Aigner
F
,
Schlachetzki
F
, et al
.
Inventory of a neurological intensive care unit: who is treated and how long?
Neurol Res Int
.
2015
;
2015
:
696038
. doi:
11
Suarez
JI
,
Zaidat
OO
,
Suri
MF
, et al
.
Length of stay and mortality in neu-rocritically ill patients: impact of a specialized neurocritical care team
.
Crit Care Med
.
2004
;
32
(
11
):
2311
-
2317
.
12
Harris
PA
,
Taylor
R
,
Thielke
R
,
Payne
J
,
Gonzalez
N
,
Conde
JG
.
Research Electronic Data Capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support
.
J Biomed Inform
.
2009
;
42
(
2
):
377
-
381
.
13
Knopman
DS
,
Roberts
RO
,
Geda
YE
, et al
.
Validation of the Telephone Interview for Cognitive Status-modified in subjects with normal cognition, mild cognitive impairment, or dementia
.
Neuroepidemiology
.
2010
;
34
(
1
):
34
-
42
.
14
van Swieten
JC
,
Koudstaal
PJ
,
Visser
MC
,
Schouten
HJ
,
van Gijn
J
.
Interobserver agreement for the assessment of handicap in stroke patients
.
Stroke
.
1988
;
19
(
5
):
604
-
607
.
15
Wilson
JT
,
Pettigrew
LE
,
Teasdale
GM
.
Structured interviews for the Glasgow Outcome Scale and the extended Glasgow Outcome Scale: guidelines for their use
.
J Neurotrauma
.
1998
;
15
(
8
):
573
-
585
.
16
Nelson
LD
,
Ranson
J
,
Ferguson
AR
, et al
.
Validating multi-dimensional outcome assessment using the TBI Common Data Elements: an analysis of the TRACK-TBI Pilot study sample
.
J Neurotrauma
.
2017
;
34
(
22
):
3158
-
3172
.
17
Herdman
M
,
Gudex
C
,
Lloyd
A
, et al
.
Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L)
.
Qual Life Res
.
2011
;
20
(
10
):
1727
-
1736
.
18
Golicki
D
,
Niewada
M
,
Buczek
J
, et al
.
Validity of EQ-5D-5L in stroke
.
Qual Life Res
.
2015
;
24
(
4
):
845
-
850
.
19
Chaisinanunkul
N
,
Adeoye
O
,
Lewis
RJ
, et al
.
Adopting a patient-centered approach to primary outcome analysis of acute stroke trials using a utility-weighted modified Rankin Scale
.
Stroke
.
2015
;
46
(
8
):
2238
-
2243
.

Footnotes

To purchase electronic or print reprints, contact the American Association of Critical-Care Nurses, 27071 Aliso Creek Rd, Aliso Viejo, CA 92656. Phone, (800) 899-1712 or (949) 362-2050 (ext 532); fax, (949) 362-2049; email, reprints@aacn.org.

Pilot phase data were initially presented at the Neurocritical Care Society’s annual meeting in National Harbor, Maryland, in 2016, and implementation phase data were presented at the Neurocritical Care Society’s annual meeting in Kona, Hawaii, in 2017.

 

Financial Disclosures

None reported.

 

See Also

To learn more about neurocritical care, read “Safety of Continuous Peripheral Infusion of 3% Sodium Chloride Solution in Neurocritical Care Patients” by Jones et al in the American Journal of Critical Care, 2017;26(1):37-42. Available at www.ajcconline.org.