The Second Time Around

In our roles as clinicians and researchers, many of us rely on secondary sources to provide succinct summaries of research and expert commentary about research results. Use of secondary sources simplifies the complex tasks associated with locating and interpreting relevant research, staying current in a broad discipline as well as specialty practice, and incorporating research into practice. When done well, secondary sources can augment our appreciation of primary research literature. Topical literature reviews can provide overviews of related research in a particular area. The introduction and discussion sections of primary research reports routinely cite and interpret the work of other researchers to provide rationale and context for the study, and study findings are framed in the context of other relevant studies.

Health news coverage is a prevalent secondary source of information about new research. Such news coverage may be offered by professional news organizations, and many news outlets have reporters who specialize in health related topics. Health care and clinical specialty organizations also cover research-related news; the American Association of Critical-Care Nurses’ Bold Voices magazine regularly covers research news, and the Clinical Pearls section of the American Journal of Critical Care (AJCC) provides clinically useful synopses of research (secondary sources) that link directly to the research articles (primary sources) in the same issue.

Despite the usefulness of secondary sources, we should exercise caution about overreliance on them. Reliance solely on secondary sources may shape our perceptions about research results in ways that do not agree with what primary sources actually reported. Several kinds of errors can occur in secondary sources. Factual errors occur when primary source data (for example, study sample size) are incorrectly reported by a secondary source. Secondary sources cannot include all of the information found in the original primary source, and what is omitted can change readers’ understanding. For example, a secondary source may fail to describe a research study’s focus on sample characteristics, and readers may assume that the results apply equally well to their patients when that may not be true. Original sources may be misinterpreted or paraphrased incorrectly. The original meaning of direct quotes may be altered if they are presented out of context or subjected to truncation.

Research studies about citations often classify errors in secondary sources as “minor” or “major.” Minor errors do not alter the meaning of the original source. For example, misspelling an author’s name in the description of a research study may make retrieval of the primary source more difficult, but is not likely to affect the interpretation of the findings. Major errors are those that substantially change the message of the primary source, and these are of greatest concern in scientific literature.

A recent case study by Cleary and colleagues in Nurse Author and Editor²  illustrates the dangers of overreliance on secondary sources and the magnification of errors perpetuated by indirect references (that is, referencing what a secondary source says about a primary source, rather than referencing the primary source directly). The case study follows the fate of a “Letter to the Editor” by Porter and Jick published in the New England Journal of Medicine in 1980.³  The letter, which is a brief paragraph long, is reproduced in its entirety in the case study, so that readers can see the primary source as Cleary and colleagues trace its citation history. Based on a computer query of data collected over 2 decades during the 1960s and 1970s in the Boston Collaborative Drug Surveillance Program, Porter and Jick concluded that new narcotic addiction was rare in hospitalized medical patients receiving narcotics.

Astonishingly, the letter had been cited 896 times by April 2016 when the case study was published; many of these citations referenced other secondary sources and offered interpretations that were farther and farther afield from the original letter’s design, data, and conclusion. The body of literature that grew up around this letter influenced health policy for decades. Cleary and colleagues found secondary sources describing the letter as a “landmark report” and “an extensive study.” Cleary and colleagues had this to say, “Like the old-fashioned game of telephone, one research report quoted the next, and it seems that no one bothered to do the essential work of looking up the original citation to see what it said and reference it accurately.”² 

So how do we balance the clear benefits provided by secondary sources and the obvious need for caution? Editors, authors, and readers all have parts to play.

As editors, we take the accuracy of citations as an inviolable trust, and AJCC has robust processes in place to ensure that what is published in the journal is accurate and complete. As editors, we carefully read every manuscript, and our processes are bolstered by highly competent peer reviewers. AJCC peer reviewers are experts who are familiar with the primary sources in their specialty areas. It is not uncommon for reviewers to suggest that authors consider specific primary sources, or for peer reviewers to challenge authors’ interpretations of primary sources. The interactions among editors, peer reviewers, and authors as manuscripts are revised prior to publication are essential for ensuring accuracy. The managing editor and copyeditors of AJCC are meticulous in manuscript production, so that the number of minor errors is minimized.

Authors are central to the integrity of secondary sources. Many writing manuals, including those by the American Medical Association, American Psychological Association, and the Modern Language Association, caution authors against using indirect references. These manuals urge authors to use secondary sources sparingly, and to clearly identify secondary source citations. When citing primary sources, authors are obligated to read the entire paper, rather than relying on the abstract! Further, authors should strive to be aware of and curb their own biases as they summarize and critique research. Author opinions should be clearly identified as such.

Readers should evaluate the trustworthiness of secondary sources and use the highest quality sources. Obtaining information from more than one source can help readers identify potential errors or biases (assuming, of course that indirect referencing is not involved). Finally, reading the primary sources of research that are the crucial underpinnings of our practice is the best way to build a strong foundation for evidence-based practice.