Rapid Mortality Review in the Intensive Care Unit: An In-Person, Multidisciplinary Improvement Initiative

A main focus of the health care quality movement involves efforts to improve patient safety in intensive care units (ICUs).^1,2  Although it has been more than 20 years since the Institute of Medicine’s 1999 report sparked mainstream attention and incited significant allocation of resources to the issue of preventable mortality,^3,4  reducing patient harm remains a complex and ambitious goal for the international medical community.^5,6  In particular, remaining challenges include not only identifying preventable deaths and areas where care could be improved, but also leveraging them to effect system-wide change.^7,8 

Various strategies to efficiently and comprehensively identify areas for improvement have been investigated with regard to deaths in the ICU.^9–11  Retrospective case reviews, in which trained practitioners examine a deceased patient’s medical record, are perhaps the most common tactic, yet such reviews are limited by their dependence on medical documentation, discrepant definitions of preventability, and poor interrater reliability.⁹  Researchers have also used electronic surveys of frontline providers, with positive results, though the usefulness of survey results greatly depends on provider response rates, timeliness, and candor.¹⁰  Finally, although structured morbidity and mortality conferences can be a useful forum for discussions regarding quality and safety optimization,¹¹  they are unlikely to provide an efficient and practical means of reviewing all patient deaths.

We aimed to assess whether a structured, in-person “rapid mortality review” (RMR) with providers regarding deaths in the ICU would be a useful means of identifying preventable deaths and action items that could lead to improvements in care. By engaging a multidisciplinary team of frontline clinicians, including house staff, intensivists, and critical care nurses, in person and in near real time, we hypothesized that this novel means of reviewing deaths in the ICU would be effective in allowing us to understand both individual- and systems-level issues that contribute to mortality. Our goal was to assess the capability of this RMR tool to identify instances of preventable death, patient care concerns, and action items for improvement, with the ultimate aim of optimizing the delivery of patient care within our hospital system.

Our institution’s RMR was developed as a Department of Medicine quality improvement initiative and was granted an exemption by the University of California, Los Angeles Institutional Review Board. This study adhered to the Standards for Quality Improvement Reporting Excellence guidelines.¹² 

Our medical ICU is a 24-bed closed unit within the Ronald Reagan UCLA Medical Center. Each week, the ICU’s clinical nurse specialist sends a list of patients who died to the Department of Medicine’s quality team. The ICU residents, fellows, and attending physicians who were involved in these patients’ cases are identified through chart review and invited via email to attend that week’s RMR. Consultants who were involved in the case are also welcome to attend the session. Our RMR process was pilot tested in 2013 with only a subset of patients who had died; this subset gradually increased to a convenience sample of deaths that occurred during a portion of the week. By 2017, the team was attempting to review during the RMR all deaths that had occurred in the ICU during the past week.

The in-person RMR meeting takes place in a private conference room in the ICU, lasts about 30 to 45 minutes, and occurs on the same day and at the same time each week. In addition to physicians on the ICU team, a chief medical resident and either the clinical nurse specialist from the ICU or the director of the nursing unit also attend. A quality program facilitator (either an attending physician, chief medical resident, ICU fellow, or quality-trained nurse) leads the meeting; this person was not directly involved in caring for the deceased patients. This facilitator also reviews the cases beforehand to increase efficiency during the discussion.

For each patient death, the facilitator leads a semistructured interview with the care team; during these 10- to 15-minute-long discussions, the team explores the details of the case, focusing on care challenges, teamwork, handoffs, end-of-life planning and care, and areas for systems-related improvement. The facilitator’s primary focus centers on questions such as, “Was the death potentially preventable?” and “Are there any aspects of care that could have been improved?” Questions that the care team answers in the affirmative are followed by additional open-ended questions.

After the meeting, the facilitator categorizes the issues discussed and records them in the encrypted Research Electronic Data Capture database.¹³  On the basis of the results of both the in-person discussion and the chart review, the facilitator makes a comprehensive designation regarding the preventability of the death and action items for improvement. Two members of the quality team meet weekly to review the data entered into the Research Electronic Data Capture database and make appropriate referrals for the department and the health system. The database is continuously updated to clarify whether the action items have been completed. This review protocol remained the same throughout the duration of the study.

Our analysis included 3 steps: first, we categorized action items, then we evaluated whether action item resolution led to systemic change. Last, we used statistical measures to analyze the data. Each action item entry was designated as 1 of 18 categories that are often cited in the literature (see Table 2 for the full list of categories). When examining completed action items to determine whether tangible system-wide change occurred as a result of the action item, we coded an item as “leading to systemic change” only if 2 members of the quality team agreed that hospital-wide change occurred as a result of the item.

We used Stata 14 software to analyze the data, which were deidentified before analysis. Categorical variables are reported as counts and percentages, and continuous variables are summarized as means and standard deviations. We analyzed results using the χ² test for categorical variables and the Student t test for continuous variables. Bivariate analyses were done to detect associations with the formation of an action item. All P values are 2-sided, and we considered a P value less than .05 to be statistically significant.

During a 5-year period (November 2013 through October 2018), the RMR team reviewed 542 deaths. Baseline characteristics of these cases are presented in Table 1. The 391 deaths that occurred after the initial pilot study constituted 82% of all of the deaths that occurred in the ICU during the study period. In most instances, an intensivist (either an attending physician or a fellow) facilitated the review; a hospitalist, chief medical resident, or quality-trained nurse facilitated the remaining 35% of cases. Although most patients (64%) had been admitted through the medical center’s emergency department; the rest had been transferred from an outside hospital, from a clinic, from home, or from a skilled nursing facility (Table 1). The length of stay ranged from 0 to 678 days (mean, 19.4 days). At least 1 consulting service was involved in 513 cases (95%).

Almost three-fourths of patients transitioned to comfort care before dying, and the treatment teams deemed death to be “expected at the time of death” in all but 5% of cases. Discussions about the goals of care had been started in the outpatient setting in less than 10% of cases. Acute medical complications such as myocardial infarction, respiratory failure, or stroke were the primary reasons for mortality in the majority of cases. An autopsy was requested in relatively few cases (Table 1).

Only 7% of deaths were deemed potentially preventable (as determined by the treatment team, the facilitator, or both). However, the treatment team believed that care could have been improved for almost half of the patients, and the RMR facilitators identified issues in more than 50% of the cases (Table 1). Such issues included problems or concerns with communication or teamwork, advance care planning, systems (eg, lack of protocols, resource availability, throughput), delays in care, medical errors, procedural complications, and hospital-acquired infections. Despite these issues, only 6% of cases were referred to the departmental morbidity and mortality committee for peer review. Providers believed that the attending physician had been appropriately involved in all but 23 cases (4%).

Facilitators identified a total of 253 concrete action items in 175 cases (32%) (Table 2). The most common categories of action items were review or development of hospital policies, enhancement of provider education, and improvement in interactions with consultants. Of all the action items, 60% were completed; 11% of those (n = 29) led to tangible systemic change. Examples of completed action items include the creation of a standardized checklist for nurses to complete in the transfer center before releasing a bed for a patient transferring in from an outside hospital, optimization of the “Orders” tab in the electronic health record so that 1-time orders are separated from continuing orders, development of a formalized supervision policy for house staff who are on call overnight in the ICU, and clarification of the overhead “code blue” call process with the hospital communications team. Examples of incomplete action items include developing a best practices document for the use of paralytic drugs in the ICU, formalizing a plan for joint ICU-oncology rounds for critically ill bone marrow transplant patients, and improving processes to expedite the initiation of hemodialysis for patients who need emergent treatment. In addition, some action items identified ambitious quality improvement projects that are ongoing.

Action items were more likely to be identified in cases where the patient was not on comfort care, where cardiopulmonary resuscitation (CPR) was performed, when the death was unexpected, and when a nonintensivist was the RMR facilitator (Table 3). Deaths that were deemed “potentially preventable,” deaths of patients for whom providers believed that care could have been improved, and deaths of patients in whose cases the attending physician had not been adequately involved were also more likely to generate action items. Finally, deaths that resulted in any identified issue or concern about patient care (as determined by the facilitator), as well as deaths raising specific concerns regarding medical errors, care delays, communication or teamwork, procedural complications, systems issues, and advance care planning were more likely to lead to action items. Patient age, sex, length of stay, source of admission, cause of death, and timing of discussions about goals of care were not associated with the generation of action items.

In this 5-year analysis, we found that an in-person, multidisciplinary RMR process in the ICU effectively helped providers identify not only preventable deaths and patient care concerns, but also concrete action items for improvement. Despite a relatively low rate of preventable mortality, concerns related to patient care were identified in more than half of the cases. Furthermore, action items were not only identified but also completed in most of these cases. These findings suggest that a short and timely in-person discussion with the multidisciplinary team of frontline providers can be a powerful tool for efforts to both improve quality and prevent mortality in the ICU.

Various other mortality review protocols have been proposed in the literature, including retrospective chart reviews, provider surveys, and morbidity and mortality conferences.^9–11  Our process fundamentally differs from these protocols in its focus on an in-person discussion with numerous members of the multidisciplinary care team. Face-to-face interviews, which are often considered the gold-standard survey mode, provide clear benefits: they afford a flexible, semistructured interview with the participant(s), they offer the potential to build rapport sponse and the inherent bias that accompanies it.¹⁵  The main drawback of face-to-face interviews—the large time commitment—was also minimized by our protocol’s streamlined nature and “rapid” focus. The feasibility of our process is highlighted by our ability to review more than 80% of all deaths that occurred in our ICU. Further, the most common reasons for not reviewing deaths were when clinical acuity prevented meeting attendance or when the date of the RMR fell on a holiday, as opposed to provider reluctance or time limitations. These reasons suggest that the cases the teams did not review did not inherently differ from those they did review, and thus the unreviewed cases probably would not affect the results.

Our experience suggests that this short, multidisciplinary interview can yield valuable insights on both death preventability and areas for hospital improvement projects, especially when compared with other mortality review protocols. For example, whereas studies in which independent providers performed retrospective chart reviews identified comparable rates of “potentially preventable” deaths (ranging from 4.5% to 22.7%),^9,16,17  those reviews identified opportunities for improvement in only 8.5% to 23% of patient deaths^18,19 —a rate much lower than the 32% identified through our method. A study that used an alternative mortality review method in which primary providers were surveyed via an electronic questionnaire yielded a preventability rate of 4.8% and elicited suggestions for improvement in only 8% of deaths.¹⁰  The fact that our study detected a similar rate of preventable mortality (7%), while simultaneously identifying much higher rates of facilitator-identified and provider-identified care concerns (54% and 42%, respectively), as well as a larger percentage of “action items for improvement” (32%), suggests the proficiency of our method at both unearthing systemic issues and also soliciting opinions for constructive improvements in care.

The primary benchmark for reviews, however, should not be solely the number of action items identified, but also the quality and validity of those items. Notably, the quality of action items identified through our process seems to be high. First, despite the subjectivity and biases inherent in any retrospective analysis,²⁰  action items were more likely to be identified in cases in which the patient’s death had been preventable and cases in which providers had care-related concerns. This lends credence to the clinical relevance and potential impact of these action items. In addition, among the identified action items, more than 1 in 10 led to tangible systemic change, with 29 discrete changes occurring during the study period. This number of discrete changes is comparable to what has been reported in other successful quality improvement studies,²¹  and, given the need to balance eagerness for change with the development of informed and sustainable processes,²²  this low but significant rate seems appropriate.

Our analysis also suggests possible ways to optimize further the yield of mortality reviews. In settings where an in-person review of every death is not possible, reviewers may want to prioritize cases in which cardiac arrests occurred in the hospital and the patient required CPR, and cases in which the patient was not receiving comfort care at the time of death, as these 2 objective measures were, for us, significantly associated with a higher likelihood of generating an action item. Although in this analysis other variables such as the perceived lack of involvement by an attending physician and identified care-related concerns also significantly correlated with the generation of an action item, these elements are harder to use a priori when screening for high-risk cases, as they require direct opinions from members of the health care team and a more in-depth chart review.

It is also notable that nonintensivist facilitators were more likely than intensivist facilitators to identify action items. Although a reviewer who is more removed from the case (eg, one who is not a departmental colleague of anyone on the care team) may be more apt to identify issues, this is contrary to a study that showed that internal reviewers might be more rigorous than external reviewers in identifying preventable deaths.²³  The reason for this difference is not clear and should be investigated further, particularly given that we did not extensively characterize our facilitators and thus could assess only for associations according to their specialty and their sex.

Limitations include the use of data from only a single center, the heterogeneous population of review-ers, and the inherent subjectivity of some of our data. Our study was performed in an academic hospital with a highly complex and specialized patient population; thus our rates of preventable deaths and care improvements may not be generalizable to other hospital settings. In addition, attribution and confirmation biases may have affected facilitators’ interpretations, as most of the reviewers did not receive formal training, though these limitations are not unique to our study. Last, although we encouraged the attendance of all members of the care team, not all providers always attended, given clinical circumstances and scheduling conflicts. Because this study investigated a quality improvement effort, we did not require members of the heath care team to sign in before RMR meetings, and therefore we could not evaluate the effect of missing providers.

Strengths of the study include the long study period (5 years) and the large number of deaths reviewed. In addition, the short time lapse between death and the RMR session probably minimized recall bias, and the relatively minimal time commitment afforded by our RMR protocol enhanced feasibility from the standpoint of both the care team and the facilitator. Our RMR protocol also benefits from its team-based and multidisciplinary nature, particularly given that team-based strategies have been shown to improve the culture and outcomes related to patient safety.²² 

Our quality improvement study of 542 deaths in a 5-year period evaluated a novel in-person, multidisciplinary mortality review process that helps providers and facilitators to effectively identify both preventable deaths and areas for care-related improvements. Amidst our continual duty to improve patient safety, future studies are needed to assess whether this process can be adopted in other hospital systems and whether the systemic changes resulting from this process ultimately translate into objective improvements in care.

The authors thank the members of the quality team at the University of California, Los Angeles.