Evidence-Based Pandemonium

Any rational person would want his or her health-care to be based on the best evidence for good outcomes. (Most of the studies evaluating evidence-based healthcare come from countries and institutions that use the word “medicine” to refer to all aspects of healthcare, eg, nursing care, physician care, and anything having to do with the treatment and management of patients. When the term evidence-based medicine is used in this editorial, it by and large refers to this inclusive definition and is used in the spirit of political correctness.) While philosophically this is a fine ideal, a real problem arises when dealing with the quality and quantity of the data. What actually constitutes sufficient evidence?² 

The public, with considerable, though at times anecdotal, justification, has laid a heavy burden on the healthcare professions. This has been fuelled by the Institute of Medicine’s publication, To Err Is Human: Building a Safer Health System.³ Members of the public want greater safety in their hospitals and to know that they are being treated optimally. The economy has shrunk, but not the relative size of the slice of the pie put aside for healthcare. The number of uninsured continues to rise (now approaching 43 000 000), as do the premiums of insurance policies, which are often offering decreased benefits. During a time of greater fiscal restraints, we are being asked to do more with less. Middle management personnel are often being rewarded for ramping up “productivity,” causing exploitation of both nursing and medical staff in the name of “patient-centered care,” or just out of expediency. This, in turn, has led to the formation of bargaining units, the only reasonable way to try to protect an appropriate working environment. (See the Letters to the Editors on pages 14–16 of this issue of AJCC about the September editorial titled Professionalism.)

Some politicians seem to be jumping on the bandwagon, trying to make political capital out of patient safety, while protecting expensive, inefficient, bottom-line–oriented management and insurance systems. These systems drain much of the funding that could be used to improve patient care. Evidence-based care arose from a need to contain healthcare costs, and, during the past 15 years, much of the impetus has come from expanding managed care systems. Payment for care could be disallowed or the care could be refused if there was not a reasonable indication that it could improve outcome. Evidence to justify parsimony, based on efficacy, was needed, and the rules for obtaining it established.

There is a zeal among some proponents of evidence-based healthcare who are promoting an ideal that some greater therapeutic utopia lies within our grasp. We must follow the precepts of patient care based on evidence from the current gold standard of randomized controlled trials (RCTs) or at least derived from a meta-analysis of the available data.⁴ This ideal has been part of legal thinking for more than 2000 years, when the ancient Roman advocate Marcus Tullius Cicero put forward the principle of summum bonum (the highest good). In other words, the best legal system would produce the most good for the greatest number of citizens. This is a worthy goal for the law and certainly has a place in the management of patient care. We are now left with a serious concern that the safety and efficacy of a care plan or therapy based on what may be statistically best for most patients may not optimally fulfill the needs of a particularly complex critically ill patient in an intensive care unit and could even be deleterious.

What can well-conducted RCTs show us? Certainly, with well-controlled conditions, RCTs can demonstrate different outcomes when different treatment modalities are being compared. One of the dilemmas of the experimental conditions arises out of the inclusion and exclusion criteria. For instance, if an investigation will accept only a literate (able to read and understand a consent form based on the principles of patient autonomy and the requirements of a human subjects review committee), English-speaking patient, then a socioeconomically better-off segment of society is being used. The results of the investigation give maximally valid information only concerning the population studied in the particular study setting.

In a well-defined setting, RCTs may also be able to demonstrate the effect of introducing a new treatment or management (eg, early discharge, some “fast track” strategy, protocol for catheterization, or the routine use of antibiotics or steroids for certain conditions). There is considerable apprehension about other limiting factors that may occur in RCTs, particularly when the results of a trial are used to direct clinical practice. These are tellingly listed by John Marini in a discussion on the use of RCTs in critical care.⁵ There is need to examine “…motivation, funding, selection, design, execution, and interpretation.” In other words, we must worry about everything! These issues also have been raised in the evidence-based practice of nursing care.⁶ 

The most frequent concerns about the usefulness (U) of information are its relevance (R) and validity (V) and the work (W) needed to access it. Slawson and Shaughnessy⁷ gave these factors the following relationship: U = V × R/W.

In other words, for information to be useful, it has to be true, relevant to the patient’s needs, and easily obtainable.

In 1991, Wyatt⁸ believed that physicians used some 2 million pieces of information to manage patients and that medical information had a doubling time of approximately 19 years. Throughout the health-care community, we are being assailed by overwhelming torrents of information, and there is no easy method of sorting out what we really need. Also in 1991, the editor of the British Medical Journal⁹ pointed out that 85% of studies published in the medical literature produced data that was less than satisfactory for use as evidence by the now-established criteria for evidence-based medicine.

How can we get the information we really need to care for our patients? The questions that arise are often complex, dealing with both an individual patient’s needs and a particular area of knowledge. Physicians have tended to rely on medical experts and colleagues rather than access an evidence-based source,¹⁰ but this behavior pattern is probably changing, as the use of the Internet to access information is becoming more prevalent. Textbooks are seldom on the cutting edge of information, and journals tend to be too diffuse to provide answers to clinical problems as they arise. To obtain relevant information at the bedside, electronic, user-friendly devices that are able to access “both a large valid database of medical knowledge and the patient record” need to be developed.¹⁰ 

An initial step forward in addressing the information glut was an article about POEM¹¹ (InfoRetriever, Charlottesville, Va) that appeared in the British Medical Journal. POEM, which stands for Patient-Oriented Evidence that Matters, must meet 3 criteria: “It addresses a question that doctors encounter, it measures outcomes that their patients care about: symptoms, morbidity, quality of life and mortality, and it has the potential to change the ways doctors practise.” POEMs are the result of scanning current issues of more than 100 journals to produce the validated evidence that matters. (InfoRetriever, the company that creates POEMs, has more information about the service on its Web site: www.infopoems.com). This type of service needs to be expanded into all areas of clinical practice.

The validity issue is much tougher and is confounded by the problem areas listed by Marini.⁵ The trickiest problem is the conflict of interest, particularly since so much research is being funded by the pharmaceutical industry and proportionally less research is being funded by noncommercial or governmental sources. Few of us, during the course of our careers, have not been recipients of some commercial institution’s largesse. Research is funded, national meetings and educational endeavors receive financial support, advertising revenue makes possible the publishing of journals (including this one), and prestigious authorities (from speakers’ bureaus) are paid large honorariums to speak favorably about products and spread information. Giveaways such as pens, “free” lunches, and other such blandishments are used to help convince us of the worthiness of a product.

From the perspective of patient care, the pharmaceutical industry has been responsible during the last 60 years for the development and manufacture of almost all the new drugs—“drugs that have transformed medicine.”¹² A healthy pharmaceutical industry, able to develop and manufacture more new drugs, is essential to the progress of patient care. Drug companies need to sell their products to survive, but there is increasing concern that the marketing aspect of sales has entangled the healthcare professions to an unhealthy extent. Both parties have to work closely together and need to support each other to be optimally productive.

In May 2002, an entire issue of the British Medical Journal was devoted to an international discussion of the relationship of clinical care practice and the pharmaceutical industry. (This is well reviewed elsewhere.¹³) A striking theme was the bias that creeps into published evidence sponsored by the pharmaceutical industry,¹⁴ even in the face of full disclosure of possible conflicts of interest. Multiple publication, selective publication, and selective reporting seemed to account for this bias. Every journal editor wants to avoid being a party to the publication of biased data, but the biases are often not apparent in an individual manuscript.

In another study, Lexchin and colleagues¹⁵ examined industry-sponsored and independently sponsored investigations. It was noted that all studies were of equally high quality, but the industry-sponsored studies were 4 times as likely to favor the sponsor’s product. It would be easy to blame the pharmaceutical industry for the current state of affairs, but this would be unrealistic and unfair. We are all stuck in the dilemma of trying to get the best evidence without compromising our financial support.

Obviously, evidence-based practice is a worthy goal, but the limitations of RCTs and meta-analyses have to be overcome; possibly, these techniques will be superseded. The medical-industrial complex is necessary for the development of better patient care, and an overhaul of the relationship may be necessary to provide long-term benefit to all parties. More user-friendly means of disseminating information are needed. In the meantime, let us not throw out the baby with the bathwater!