Conundrums in the Conscientious Use of Current Best Evidence

Critical care providers are passionate about helping patients survive critical illness and mitigating the negative effects of both illness and treatments patients receive. Evidence-based practice (EBP) is a central component of patient- and family-centered care. The American Association of Critical-Care Nurses (AACN) defines EBP as “A problem-solving approach in practice that involves the conscientious use of current best evidence in making decisions about patient care … [and] incorporates a systematic search for and critical appraisal of relevant evidence to answer a clinical question….”¹ 

To say that the foundation of EBP is the conscientious use of current best evidence is to acknowledge that EBP requires the involvement of both the researchers who generate the evidence and the clinicians who are charged with translating such evidence into a clinical decision that reflects their clinical expertise and their knowledge of the patients’ values and preferences. This evidence may involve what should be done but may also involve what practices we should not do. The Choosing Wisely campaign²  is an ongoing effort to identify and reduce the use of unnecessary tests and procedures through provider awareness and discussion between providers and patients. The Critical Care Societies Collaborative (American Thoracic Society, AACN, American College of Chest Physicians, and Society of Critical Care Medicine)³  and the American Academy of Nursing⁴  have both contributed lists of practices to be avoided to the Choosing Wisely campaign. Clinical researchers want to contribute robust evidence to inform clinicians’ practice. Systems have been developed to assist clinicians in understanding the quality of the research evidence and the potential strength or trust-worthiness of a recommendation, including GRADE (Grading of Recommendations Assessment, Development and Evaluation)⁵  and AACN levels of evidence.⁶  The AACN levels of evidence rating system recognizes 2 broad categories of evidence that influence clinical practice. Experimental studies (randomized clinical trials [RCTs] or systematic reviews of multiple RCTs) are considered to be more trustworthy than nonexperimental studies (observational studies, case studies, expert opinion, untested theory). Within each category, the more rigorous the study design, the more potentially trustworthy the results.

Well-designed clinical trials provide the firmest foundation for practice and are considered the gold standard for experimental studies. When available, the best level of evidence comes from multiple well-designed controlled studies that converge on the same answer to a particular clinical problem. Reproducibility of results among multiple studies provides the greatest confidence in applying evidence.

In the critical care setting, it is particularly difficult to design and conduct a clinical trial that will inform practice. Clinical care must always be paramount—even if this compromises the scientific rigor of a study. Critical care patients are heterogeneous, and many factors can influence their response to interventions being evaluated. Trial designs must try to account for such factors as current pathology, underlying disease conditions, severity of illness, age, gender, socioeconomic status, and family circumstances and dynamics. Enrolling and obtaining consent from patients during critical illness are complex procedures: potential subjects are often unable to provide consent because of impaired capacity for decision-making due to illness or medication; the potential subject’s legally authorized representative for consent may be overwhelmed with decision-making related to the patient’s clinical care.

Two recent studies illustrate the issues surrounding generation and application of best evidence. Both were well-designed RCTs that addressed important problems affecting patients. Researchers in both studies concluded that their tested intervention(s) did not improve their outcome of interest beyond the standard of care, and yet each study may have different implications for clinical care and future research.

In light of the multiple studies showing the association between delirium and adverse outcomes in critically ill patients (eg, higher mortality, higher risk of long-term cognitive impairment), the question of how to treat delirium in the critical care setting is a pressing one. The MIND-USA (acronym for Modifying the Impact of ICU-Associated Neurological Dysfunction–USA) study was a randomized, double-blinded, placebo-controlled clinical trial that compared the efficacy of 2 antipsychotic drugs commonly used in critical care (haloperidol or ziprasidone) with placebo on the duration of delirium and coma in critically ill patients.⁷  From 2011 through 2017, informed consent was obtained from 1183 patients with acute respiratory failure or shock across 16 medical centers. Delirium developed in 566 of the patients who provided consent, who were randomized to 1 of 3 treatment groups: haloperidol (up to 20 mg daily), ziprasidone (up to 40 mg daily), or placebo. The study was blinded so that the group assignment was not known to the clinical providers, the patients, or the patients’ families. The adjusted median number of days alive without delirium or coma was similar in the 3 different groups: 7.9 (95% CI, 4.4–9.6) versus 8.7 (95% CI, 5.9–10.0) versus 8.5 (95% CI, 5.6–9.9) in the haloperidol, ziprasidone, and placebo groups, respectively. The P value for the overall effect across the groups of .26 suggested that any differences between the groups were most likely attributable to chance.

When the “negative” results of this single study are taken together with other similarly “negative” results of other RCTs^8,9  published while MIND-USA was ongoing, a positive direction for patient care emerges: antipsychotics should not be routinely administered to reduce delirium. Dr Wes Ely, lead author, suggested that routine use of the “ABCDEF” bundle (which was the standard of care offered to all patients across the 3 treatment groups in the MIND-USA study) may be a more appropriate strategy to reduce delirium in critically ill patients.¹⁰  The best available evidence thus points toward what to do (the ABCDEF bundle) and what not to do (administer antipsychotic medications) to reduce delirium.

Advances in the use of extracorporeal membrane oxygenation (ECMO) technologies to rescue patients with severe and refractory hypoxemia invites the question of whether ECMO can reduce the high mortality rates in patients with severe acute respiratory distress syndrome (ARDS) whose condition is deteriorating despite appropriate strategies of mechanical ventilation. In the ECMO to Rescue Lung Injury in Severe ARDS (EOLIA) trial,¹¹  conducted for 6 years in multiple institutions across Europe, 249 patients with severe ARDS and evidence of refractory hypoxemia or acidosis despite optimal ventilator management were randomized to 1 of 2 groups: an ECMO group (n = 124), or a control group (n = 125). The ECMO group began receiving ECMO immediately after randomization and the control group received the standard of care as practiced in Europe, which included low-tidal-volume ventilation and modestly high positive end-expiratory pressure for alveolar recruitment, with physicians strongly encouraged to use neuromuscular blocking agents and prone positioning. In that study, the 60-day mortality rate was 35% in the ECMO group and 46% in the control group (for a relative risk of 0.76 [95% CI, 0.55–1.04]). The P value of .09 suggested that differences between the 2 groups could very well be due to chance. So, this trial was yet another “negative” trial.

However, several other factors make it difficult to know how best to use the EOLIA trial results to inform clinical practice. The nature of the intervention made it impossible to blind the clinical providers to the patient’s group assignment; placebos may be administered to protect blinding in a medication study, but “placebo ECMO” is not an option! The treating physicians, who were aware of the subject’s assignment, could request that control subjects who had refractory hypoxemia receive ECMO, and 35 of the 125 control subjects (28%) did cross groups and receive rescue ECMO. The control subjects who received ECMO were very ill, and their mortality rate was higher than that in subjects in either the ECMO or control group. The investigators highlighted the tension between clinical care and research rigor, stating, “This crossover rate makes it difficult to draw definitive conclusions regarding the usefulness of ECMO for severe forms of ARDS … investigators felt that it would have been unethical to prohibit crossover to ECMO in patients with very severe hypoxemia.”¹¹ 

Further complicating the interpretation, the trial was stopped before the planned number of subjects had been recruited.¹²  The study’s sample size goals assumed that ECMO would reduce mortality of ARDS from 60% to 40% (which is a preposterous 20% absolute risk reduction or a relative risk of 0.67). As such, the study’s data safety monitoring committee, which examined the data at planned intervals, was empowered (and ultimately chose) to stop the study after 75% of the sample was enrolled because the lack of a statistically significant difference between the 2 groups could be predicted from the subjects already enrolled.¹¹ 

How then are clinicians to interpret the 11% lower mortality in the ECMO group, particularly considering the high crossover in the control group and the P value of .09? The conventional (frequentist) approach to statistical analysis of such trials is to estimate the probability of the null hypothesis given the data. With this approach, the P value greater than .05 means that we cannot reject chance as an explanation of this effect. Study investigators subsequently reanalyzed the results of the study using a Bayesian approach, which generally aims to provide estimates of a probability of a hypothesis (in this case, the probability that ECMO reduces mortality in patients with severe ARDS).¹³  The results of this reanalysis suggested that if one assumed 4% absolute mortality reduction to be a clinically meaningful difference, the “posterior” probability that ECMO lowered mortality ranged from 63% to 95%, depending on whether one was strongly skeptical or strongly enthusiastic about the use of ECMO in severe refractory ARDS before this study.¹³ 

Absolute answers in critical care research are rare. Current best evidence is more frequently a volatile entity that necessarily changes as more evidence accumulates. This situation should not be cause for despair among clinicians and researchers, but rather a call to revise our thinking as better evidence becomes available. Conscientious use of current best evidence is a worthy goal, and our patients and their families deserve no less!