Discussing Evidence in the Fog of the Pandemic

The first reports of a novel coronavirus (COVID-19) infection emerged near the end of 2019 from Wuhan, in the Hubei province of China.¹  By March, leaders of the World Health Organization had declared COVID-19 a pandemic on the basis of the uncontained spread of the infection across many countries. Clinicians are called to care for patients in the face of much fear and uncertainty. As Catherine Barnette wrote in her poem “Epistemology,” we all want to “feel a little less, know a little more.”² 

What we know and how we know it will be put to the test more than ever during these times. Already, a PubMed search for COVID-19 reveals more than 5000 articles either in press or already published since the beginning of the crisis. And yet, in the face of so many publications, it will be important that we think critically about which publications are presenting new knowledge and which are not; it will be important that we think critically about which new knowledge is ready for application to clinical practice and which is not.

The controversy surrounding the use of hydroxychloroquine in COVID-19 is emblematic of the challenge of discerning what we know. On the basis of in vitro data that suggested hydroxychloroquine had viral suppression activity against coronaviruses,³  some clinicians began prescribing hydroxychloroquine for COVID-19 patients and publishing anecdotal impressions of its effectiveness.⁴  In an effort to examine this more systematically, several studies were started that aimed at testing the potential efficacy of hydroxychloroquine in patients with COVID-19. Recently, an article about an open-label, nonrandomized trial that tested the efficacy of hydroxychloroquine plus azithromycin at reducing viral clearance was published ahead of print.⁵  The study had several flaws, however. First, it was not a randomized controlled trial, and the 16 participants included in the control group differed from the 20 patients included in the treatment group; patients who refused to participate in the hydroxychloroquine intervention were included in the “controls”; patients from 3 other centers were also added as controls whereas all treated patients were from a single center. Second, investigators were not blinded to the treatment allocation, and they did not describe the cointerventions the patients received during the study. Third, the patients were not analyzed in the groups to which they were assigned: patients who got hydroxychloroquine but transferred to an intensive care unit were excluded from the analyses; patients who got hydroxychloroquine but later died were excluded from the analyses; patients who got hydroxychloroquine but stopped the treatment because of symptoms or were discharged from the hospital before the study end point were excluded from the analyses. The study reported that by day 7, 70% of the patients analyzed within the hydroxychloroquine treatment arm had negative results on a viral assay (polymerase chain reaction [PCR]) compared with 12.5% in the group that had refused the study drug. In discussing their results, the authors boldly suggested that “COVID-19 patients be treated with hydroxychloroquine and azithromycin to cure … infection and to limit the transmission.”⁵ 

Even before the open-label trial of hydroxychloroquine was published, political leaders in the United States and pundits were expressing their excitement about hydroxychloroquine’s potential to be a “game-changing” treatment for COVID-19. Many medical centers started to protocolize the use of hydroxychloroquine in their treatment protocols for COVID-19. We believe that such a rush to systematically implement treatments before they have been rigorously tested will not serve our patients well. Even in pandemic times, randomized controlled clinical trials (RCTs) should still be the gold standard approach for determining the efficacy of a drug treatment.

If we rush to overuse hydroxychloroquine for COVID-19 patients on the basis of flawed data, we risk exposing our patients to known complications of the drug such as ventricular arrythmias or sudden cardiac death.⁶  With the overuse of hydroxychloroquine for COVID-19 patients, patients who depend on the drug to control their autoimmune diseases (for which the drug is approved by the Food and Drug Administration) have experienced trouble getting access and skyrocketing prices. If in choosing to use hydroxychloroquine in COVID-19 patients, we neglect to rigorously test other potentially effective treatments, then we will have missed an opportunity to impart new knowledge to the care of future patients with COVID-19.

Indeed, there are real challenges to conducting rigorous RCTs during a pandemic. In normal circumstances, RCTs are expensive, are cumbersome, and can be too slow to diffuse results into practice.^7,8  Treatment protocols in RCTs are often not flexible enough to account for the heterogeneity (ie, subphenotypes) in disease presentation.⁹  Adequate informed consent for research participation is made more difficult during a pandemic: the emotional distress of the patients and families may preclude adequate exchange of information; in situations where patients are unable to consent for themselves, surrogates may be harder to find because most hospitals have had to limit family visits in order to decrease community spread of the infection. The high level of uncertainty and fear during a pandemic may make patients, surrogates, and clinicians more averse to the inherent ambiguity of randomization.¹⁰ 

Nevertheless, modern innovations in clinical trial design make rigorous clinical trials during this pandemic more feasible than ever before. Adaptive trials, for example, allow for changes in the study protocol over time based on new information (eg, in the inclusion/exclusion criteria or the proportion of participants randomized to a control group).⁸  Platform trial design expands the scope of a trial from one intervention to a suite of interventions, allowing comparisons between many interventions within different treatment domains for a disease or syndrome.⁸  The Randomized Embedded, Multifactorial Adaptive Platform for Community-Acquired Pneumonia (REMAP-CAP) trial is one such recent trial that is enrolling COVID-19 patients around the world to quickly answer many treatment questions.¹¹  Patients who are eligible for participation in REMAP-CAP will be randomized to receive one intervention in each category of treatment (eg, type of antibiotic, antiviral treatment, steroid use). The study protocol allows data that are already accrued in the study to be used to change the randomization ratio to increase the chance that the patient is assigned to a likely beneficial treatment. Any specific treatment arm of the study will be terminated when enough data have accrued rather than after a specific sample size is reached, which allows new questions to be quickly integrated into the same trial platform.

We recognize that clinical action based only on the highest empirical evidence will not always be feasible.¹²  We will be called to act on the basis of our clinical experience and expertise. We will be called to act on the basis of our understanding of pathophysiology: of how the virus works, the pathobiology of sepsis or of acute respiratory distress syndrome. We will be called to act on the basis of our understanding of our patients (who they are and what they value). We will be asked to change what we do on the basis of systemic factors that are beyond our control such as how many ventilators are available in the hospital or whether a nurse is available to do a particular kind of dialysis in a patient with acute renal failure. However, treatment decisions based on flawed data that then become entrenched into clinical care are inherently dangerous. Our patients deserve an honest effort by researchers to generate the highest quality data on treatment efficacy. Even in the fog of this pandemic, during which much will remain a mystery, our patients deserve an honest and humble appraisal of the new medical evidence.