A checklist can be many things, as simple as “a list of items required, things to be done, or points to be considered, used as a reminder,”1 or, according to the Agency for Healthcare Research and Quality, “an algorithmic listing of actions to be performed in a given clinical setting, the goal being to ensure that no step will be forgotten.”2 Since the Institute of Medicine’s scathing report on medical error,3 checklists have gained substantial attention, owing to reports of their pivotal role in driving the success of numerous patient safety initiatives. Greater notoriety naturally precipitates greater scrutiny, however, so both the promise as well as the limitations of checklists need acknowledgement. This editorial will consider the often heralded and sometimes belittled checklist for its general benefits, potential utility, and range of evidence attesting to its value in quality and safety improvement.
General Benefits of Checklists
What is it about checklists that suggests they can be helpful to patient care? Some of the general benefits of checklists are identified in Table 1. The overriding virtues of this device may be its provision of clarity, order, and efficiency.
Usefulness of Checklists in Patient Care
The notion of harnessing the potential of checklists in patient care is virtually always ascribed to borrowing from high risk, high reliability organizations in fields such as aviation or nuclear power management,4 where complex and potentially hazardous operations need to be executed with a near zero tolerance for failures.5 In high reliability organizations, checklists have served as a “cornerstone of safety management … for nearly a century.”5(p53) Although their use in health care is more recent, performance-based checklists are familiar to any health care professional certified in cardiopulmonary resuscitation Basic Life Support by the American Heart Association within the past 5 decades.6
Today’s younger generation of critical care nurses likely encountered extensive series of clinical checklists at onboarding, when competency-based education and orientation programs guided their initial development, augmented by competency assessment programs. Baby Boomer nurses had to await introduction of competency-based education,7,8 competency-based orientation,9 and competency assessment10,11 concepts into critical care instruction in the 1980s and thereafter before performance-based checklists became commonplace for staff nurse and preceptor development.12,13 As these references suggest, I have had a longstanding affinity for the pragmatic value of checklists.
Two physicians who appear to share my bias toward checklists are Dr Gawande and Dr Pronovost. In The New Yorker, Dr Gawande related a story of how a critical care physician, Dr Provonost, drafted a 5-item checklist for preventing central venous catheter (CVC) infections at Johns Hopkins Hospital in 2001 and then asked his intensive care unit nurses to document whether physicians completed each step. Finding steps skipped in more than one-third of insertions, Dr Provonost secured administrative approval for nurses to stop physicians who skipped steps and to inquire daily whether CVCs could be removed. Over the next year, the 10-day central catheter infection rate dropped from 11% to 0%, with only 2 infections over the next 15 months.14 Dr Gawande went on to write The Checklist Manifesto.15 Dr Pronovost spearheaded stunning achievements in preventing infection throughout Michigan and worldwide.16
Evidence on the Usefulness of Checklists
Existing literature related to the effectiveness of checklists in preventing infection and improving patient safety and other aspects of health care includes an array of outcomes. This section provides an overview of each category of outcomes with offered explanations for each.
Positive Outcomes
A small sample of the large volume of research evidence reporting significant beneficial outcomes from employing checklists to improve patient care and safety is summarized in Table 2. In patient safety, Pronovost’s landmark success preventing CVC infection throughout Michigan16 ignited international impetus in using checklists for other aspects of care, most notably in the Surgical Safety Checklist of the World Health Organization (WHO).18,19 A reasonable takeaway from this checklist is the verified potential that such a device offers in saving lives and minimizing life-threatening morbidity.
When checklists are effective, at least part of the credit is owed to their design. In much the same way that designing instructional programs requires knowing the content and the learner, strategies for effective checklist design require knowing the task and the user (Table 3).
Negligible or No Differences in Outcomes
A number of studies were not able to replicate Pronovost’s central line–associated bloodstream infection (CLABSI) prevention16 or the WHO Surgical Safety Checklist success,18 finding negligible or no differences in targeted outcomes and, occasionally, negative associations. The most often cited failure to replicate CLABSI prevention16 is the “Matching Michigan” initiative in the United Kingdom, involving more than 200 intensive care units over a 2-year period.34 A number of explanations have been offered for these contrasting findings, including that the UK version was not a true replica, but varied in many ways such as numerous sites with low compliance, particularly among senior physician staff,35 a problem highlighted in other studies reporting no effects.22,36
For the WHO checklist, the dramatic reductions in both operative mortality and complication rates achieved by Haynes et al18 were not replicated in Ontario, Canada, where more than 101 acute care hospitals and 200 000 surgical procedures were examined following mandated use of that checklist.37 One explanation offered for those results related to a recurring culprit: despite the mandate, actual compliance varied within and among institutions.37
In 2015, a dual disappointment was reported when a different research group in Michigan, using a surgical checklist modeled after that of Pronovost, found no improvements in infections, complications, or mortality among nearly 65 000 general surgery patients in 29 hospitals.38
Inconsistent Outcomes
Inconsistent findings were revealed in studies in which checklists were and were not associated with significant differences in outcomes of interest. For example, in a Netherlands study,24 improvements in operative mortality were demonstrated only in patients for whom the WHO surgical checklist was fully completed; no improvements were found when the checklist was partially or not completed.
A recent systematic review of 25 WHO Surgical Safety Checklist studies found that complication rates were measured in only 20 studies, were decreased in 10, but increased in 1. Of the 18 studies that examined operative mortality, rates decreased in 4, increased in 1, and decreased only in developing nations.39
Negative Outcomes
In addition to the negative findings just noted,39 an earlier randomized trial reported a higher surgical site infection rate (45%) in the extended arm of the study versus 24% in the standard arm (P = .003).40 So how do we reconcile such contrasting findings related to the effectiveness of checklists?
Accounting for Variability in Findings
Employing checklists in safety and quality improvement programs involves much more than just a good checklist. This may be most readily apparent by using the Pronovost study.16 The checklist in that study actually comprised a bundle of 5 evidence-based actions to prevent CLABSI infections41 :
Proper hand hygiene
Chlorhexidine skin antisepsis
Maximal sterile barriers during insertion
Subclavian (versus femoral) preferred insertion site
Daily check to discontinue unnecessary central catheters
This safety program always required full compliance with all 5 elements, as well as creation of an environment conducive to continual improvement that the research team characterized as a “culture of safety” that recognizes sociocultural features of how organizations operate and people function best within them. Without the culture of safety, the checklist is not effective. One particularly insightful enumeration of those sociocultural prerequisites is summarized in Table 4, with a reminder that it took 9 months of arduous preplanning in Michigan before launching the checklist.42 These elements simultaneously created a foundation upon which the checklist can be embedded for support and circumvented barriers that could have negated the effectiveness of the checklist.
When these culture of safety considerations are included in the equation, the reasons why many safety programs involving checklists were/are not successful can be more readily understood. Analysis of the Matching Michigan study revealed marked differences in implementation and attention to sociocultural features (consensus-building, leadership support, revising local practices did not happen at many sites).35 Absent attention to those sociocultural necessities, the work culture does not translate checklist expectations into work performance standards,42 and neither staff nor their work unit makes the investment needed to achieve successful outcomes.43 Lack of buy in, particularly among senior physicians, remains an obstacle to success of these programs.44
Other analyses of WHO surgical checklist reports implemented across hospitals in England reinforce the pivotal role that specific cultural factors play in program success. Some are painfully clear and direct: If staff do not correctly, completely, or consistently use the checklist, its value is nullified and the program is destined for disappointing outcomes.36,45 Similarly, when surgical staff at participating UK hospitals were asked about barriers to implementing this checklist, the barrier most often identified was active resistance of senior clinicians.46 Other obstacles were imposing the checklist without introduction, training or support to staff; finding it poorly worded, redundant, or time-consuming; and neglecting to integrate it into existing procedures.46 Conversely, facilitators’ suggestions to improve implementation echoed many culture of safety requisites: enabling staff to tailor the checklist to local procedures; providing staff education, data feedback, and logistical support (supplies, products); supporting leadership; and instilling accountability.46
Another suggestion noteworthy when high compliance rates are reported yet no differences are found was incorporating direct observation of compliance. In a study with checklist compliance documented at 100%, observers identified that on average, hospital staff completed only 4 of WHO’s 13 requirements.47
Lessons Learned
Health care facilities that use checklists to improve patient safety and quality of care need to recognize that checklists by themselves do not improve care, but when they identify a series of evidence-based strategies, they can be employed as tools to support practices among staff individually and collectively committed to making them work. Just as the sociocultural landscape can foster improved program practices, however, it is equally capable of thwarting and undermining them at any and all stages of their life.
Developing a great checklist and expecting it to improve care is much like having healthy seedlings and expecting them to bear fruit; neither will thrive without first preparing its foundation for growth; nurturing and supporting its progress with others; removing predators that could choke its development; and monitoring its fruit at the right times.
Closing
No one ever improved patient care just by making check marks on a piece of paper or tablet, but a well-designed checklist in a supportive environment with committed practitioners can save lives, prevent complications, and improve safety. In addition to noting the compendium of checklists now offered by the Joint Commission48 and American Hospital Association49 toward that goal, please do not forget the bonus from using checklists in clinical decision support mentioned in my February editorial50 : the virtual elimination of implicit bias such as that associated with gender or race. Providing improved and safer care to all patients, rather than just to some—wouldn’t that be a welcomed serendipitous finding!