Evidence-Based Backlash: The Tale of Drotrecogin Alfa

During the past couple of decades, there has been an explosion in the molecular understanding of sepsis. Unfortunately, this understanding has not translated into pharmacotherapeutic interventions that benefit patients. However, researchers found initial positive results with the use of activated protein C (APC) for severe sepsis syndrome. Data from animal models indicated that APC might be beneficial in patients with severe sepsis via 3 primary mechanisms: reducing coagulopathy, allowing natural fibrinolysis to occur, and through innate anti-inflammatory properties.¹  But all of this would remain a hypothesis until a large, multicenter, randomized controlled trial (RCT) could be performed.

The landscape changed in 2001 when the results of the PROWESS (protein C worldwide evaluation in severe sepsis) trial were published.²  In this study—a large multicenter RCT—a clinically small but statistically significant improvement in mortality was found in the group that received APC in addition to standard therapy for sepsis (ie, aggressive but appropriate fluid resuscitation, appropriate broad spectrum antibiotics, and timely source control).

However, when the FDA went on to approve the agent for human use, the panel, as part of a split decision about whether to approve the drug at all, moved to make it dependent on an individual patient having “severe sepsis with a high risk of death.” The panel defined a high risk of death as either an APACHE II score of 25 or more or 2-organ failure. This standard was based primarily on their post hoc analysis revealing that the mortality benefit was found only in subjects who were in the upper 50th percentile of APACHE II scores. This was a very concerning move for at least 2 reasons. First, the APACHE II score was not designed to be used in this fashion. Second, the study itself was not prospectively focused on these outcomes. One of the provisos of the approval was that another study be performed (eventually called ADDRESS) that would consider those patients with severe sepsis and low risk of death.

It was an exciting time in the critical care community. The many years of basic science research in sepsis mechanisms had finally paid off and led to a drug that was proven in a large, multicenter RTC to be associated with improved outcomes in severe sepsis and septic shock. But many controversies ensued. First, there were concerns about how the trial was performed, with changes being made to the inclusion and exclusion criteria during the trial itself.⁴  Second, recommendations were made to have protocols for APC integrated into hospital sepsis bundles after only a single RCT.⁵  Third, significant concerns were raised regarding the fact that Eli Lilly and Company (the makers of APC) helped provide funding for both the initial version of the Surviving Sepsis Campaign guidelines as well as the VERICC (Values, Ethics and Rationing in Critical Care) Task Force, with the implication that physicians were rationing the use of APC based on the cost of the agent.⁶  This circumstance culminated in some high profile academic debate in the medical literature.⁷ 

The most significant problem, however, was that results of other trials of APC did not appear to be as promising as those of PROWESS. Specifically, trials focusing on patients with severe sepsis and low risk of death and on children with severe sepsis were unable to demonstrate any real mortality benefit, and the FDA indications were modified to reflect this fact. It was becoming clear that studies following the PROWESS trial replicated the initial bleeding risk without being able to replicate the mortality benefit.^3,8  In addition, the initially hypothesized mechanisms of action of APC (from animal models) could not be confirmed in humans.⁸ 

Given the lackluster results of the trials of APC in patients with severe sepsis and low risk of death (as well as children,⁹  surgical patients,¹⁰  and acute lung injury¹¹ ), the European equivalent of the FDA requested a trial of APC in patients with septic shock.¹² 

The main results of this trial—the PROWESS-SHOCK trial, unpublished as of this writing—were released in October 2011, and on October 25, 2011, the company withdrew APC from the market based on these data.^13–15  Some questions have been raised about why the mortality rate in the placebo group was so low, but that is not the point. Science is science. Data are data. If the positive results of the PROWESS trial cannot be replicated and the drug has no further indication, it should be removed. And so it was. The outcome of this trial was the final nail in the coffin of what had been one of the most promising, yet controversial, sepsis therapies in the history of critical care. Many important lessons for the practicing critical care clinician can be gleaned from this tale.

Both of the coeditors of this journal have great passion for EBM. When results of the PROWESS trial were first published, one of the authors (RHS) was completing his fellowship. It was clear at the time that the senior faculty were much more skeptical of the validity of the results than were the junior faculty and fellows. Many of us at the time assumed (incorrectly, it turns out), given all the complexities and regulations involved in designing and monitoring such a trial (not to mention the intense challenges of getting such a manuscript through the rigors of peer review at the New England Journal of Medicine), that these important initial results were highly likely to be valid and would easily stand the test of time.

The results from PROWESS, however, illustrate some of the profound limitations of EBM. An RCT such as PROWESS is supposed to be the gold standard against which all other kinds of trials are measured. A well-designed, well-executed trial with positive results like PROWESS should have been reproducible with similar positive results. In retrospect, the lack of reproducibility of these results indicated important issues: either the design of the trial was flawed, the definitions of severe sepsis are not robust enough to apply the rigors of an RCT to this patient population, or both.

On a more positive note, these results did not stand the test of time and were not reproducible so, in that sense, the scientific method ultimately worked. Though the results of a single, large RCT are important, they clearly are not sufficient for future such agents to be rapidly integrated into national guidelines or consensus statements. In addition, measurement of the use of such agents should not be used as surrogates for quality outcome in the ICU.

Another important lesson is that pharmaceutical companies should stay as far removed as possible from development of guidelines promulgated by national medical societies. One of the most important things such a society has is its reputation, which it must be careful not to tarnish. Although this can often be a great challenge, it has become quite clear from the controversies surrounding APC that the relationship between pharmaceutical corporations on the one hand and academia and national medical societies on the other should be kept distinct and transparent.

The concluding lesson is that we must remain profoundly skeptical. Clinicians who believe in EBM, as many of us do, use important positive results from RCTs as building blocks. But the single most important lesson from the rise and fall of APC is that we should maintain skepticism: maintain it until the trial can be reproduced; maintain it in the face of trusted medical societies integrating recommendations for agents before sufficient evidence is presented; and maintain it until all potential conflicts of interest have been shared. EBM is not merely one way to practice; it is the only way. In addition to understanding all of the dynamic complexities and nuances of EBM, we must develop a healthy skepticism toward new research results and apply that approach liberally as the scientific method does its important job of confirming the validity of those results.