This is an updated, comprehensive review of the psychometric properties of behavioral pain assessment tools for use with noncommunicative, critically ill adults. Articles were searched in 5 health databases. A total of 106 articles were analyzed, including 54 recently published papers. Nine behavioral pain assessment tools developed for noncommunicative critically ill adults and 4 tools developed for other non-communicative populations were included. The scale development process, reliability, validity, feasibility, and clinical utility were analyzed using a 0 to 20 scoring system, and quality of evidence was also evaluated. The Behavioral Pain Scale, the Behavioral Pain Scale-Nonintubated, and the Critical-Care Pain Observation Tool remain the tools with the strongest psychometric properties, with validation testing having been conducted in multiple countries and various languages. Other tools may be good alternatives, but additional research on them is necessary.

Pain assessment is the first essential step to adequate pain management. In the intensive care unit (ICU), pain is highly prevalent while the patient is at rest and during procedures.1,2  Assessing pain is a daily challenge in the ICU. Many patients are unable to self-report their pain for reasons such as altered levels of consciousness, mechanical ventilation, heavy sedation, and delirium.3  Although the gold standard measure of pain remains the patient’s self-report, alternative behavioral assessment tools must be used for those unable to provide a reliable self-report of pain.46 

This comprehensive review is an update of a previous review7  that evolved from the work completed on the 2013 clinical practice guidelines of the Society of Critical Care Medicine (SCCM).8  Psychometric analysis of available behavioral pain assessment tools was updated in the 2018 SCCM clinical practice guidelines4 ; new tools and validation studies published online ahead of print or published up to May 2019 were included in this review. Our aim for this comprehensive review was to analyze the development and psychometric properties (ie, reliability and validity) of behavioral pain-assessment tools for use in noncommunicative, critically ill adults.

Methods

Search Strategy and Selection of Studies

We updated the literature sources in the present review, using the same search terms and clinical databases described in our 2013 initial review7  and in the 2018 SCCM practice guidelines.4  Briefly, we used general Medical Subject Heading terms such as pain assessment, pain measurement, and the names of behavioral pain assessment tools. Searches were performed by an experienced research librarian using the following databases: Cumulative Index to Nursing and Allied Health Literature, Embase, MEDLINE, PubMed, and Web of Science. We decided to exclude Scopus because it includes a variety of academic disciplines in addition to health sciences, and the results of a Scopus search were neither specific nor relevant to the aim of this review. Studies were eligible if the population included adults aged 18 years or older in the ICU and the studies were in English, French, or any other languages spoken by the review coauthors or their graduate students. Abstracts, reviews, case reports, editorials, and letters to the editor were excluded. Titles and abstracts were screened by 2 reviewers. Disagreements about selection of studies were discussed, and consensus was reached. The majority of included papers were in English, but some were written in other languages (namely, French, Spanish, and Korean), and we had reviewers with proficiency in these languages.

Psychometric Scoring Evaluation

The psychometric scoring system was previously described7  and established with the consultation of 3 external experts in health measurement. The scoring system includes 5 main sections: (1) scale development (5 points), (2) reliability (5 points), (3) validity (10 points), (4) feasibility (2 points), and (5) clinical utility (1 point). The total score ranges from 0 to 23. Reliability refers to the overall reproducibility of measures obtained from an assessment tool, and strategies evaluated as part of the psychometric system included internal consistency (ie, homogeneity of the items) and interrater reliability. We removed the item related to intrarater reliability because it is optional when poor interrater reliability findings are found, but we described it in the presentation of each tool in Results, when appropriate. Validity refers to the interpretation of assessment tool scores. Criterion validation (ie, comparison with the gold standard measure of pain: the patient’s self-report) and discriminative validation (ie, comparison of pain scores between painful and nonpainful procedures) were evaluated. Each tool was attributed the highest score according to available study findings. However, when assumptions were not respected in the calculation of coefficient or statistical test in most studies, the score was reduced by 1 point on the specific item that was evaluated. Weighted scores were then calculated taking into account the importance of each section (ie, more weight for reliability and validity). A total weighted score was obtained by summing the weighted scores of each section and ranged from 0 to 20. The interpretation of weighted scores was as follows: very good (score 15-20), good (score 12-14.9), acceptable but other studies are necessary (score 10-11.9), and unacceptable (score < 10).7 

As described in our initial review,7  the quality of evidence for each tool was evaluated as high, moderate, low, or very low, having been adapted from the Grades of Recommendations, Assessment, Development, and Evaluation methodology used in the SCCM clinical practice guidelines process.8,9  High quality of evidence is established in a report when the scale development was well described, multiple validation strategies were tested and replicated in different studies in an overall large sample representative of the ICU population, and validation findings were consistent across most studies. Moderate quality of evidence is determined when some validation studies with an overall moderate sample size representative of some ICU patient groups were included, multiple validation strategies were used, and findings were consistent in most studies, but some methodological limitations were identified. Low quality of evidence is determined when very few validation studies were included, with an overall small sample not representative of different ICU patient groups, and some steps of scale development and validation were missing. When the psychometric properties related to the use of scale cannot be supported, a very low quality of evidence is established.

Data from studies included in the initial review of 2013 and in the 2018 SCCM clinical practice guidelines were extracted and psychometric scoring was done independently by 2 reviewers. In this updated review, data extraction and psychometric scoring of 3 new behavioral pain assessment tools were completed using a similar process. However, for new articles added in this review, data extraction was done by 1 reviewer and was checked by a second reviewer. Psychometric scoring of all tools was verified by 2 reviewers. Reviewers involved in the development of tools included in this review did not take part in data extraction of studies in which their behavioral pain assessment tool was tested.

Explanations of psychometric concepts and of the psychometric scoring system were provided to reviewers by the primary author (C.G.), who holds a doctorate in Nursing and Measurement. In addition to holding a doctorate degree, 2 of the reviewers (M. Berube, M. Boitor) completed a graduate course in health measurement. Three reviewers (C.G., A.M.J., and K.A.P.) were involved in the 2013 initial review, 3 additional reviewers participated to the 2018 SCCM clinical practice guidelines (P.M.S., J.F.P., and G.C.), and the remaining 3 reviewers (M. Berube, M. Boitor, S.S.T.) were not involved in the previous review.

Results

In the initial review,7  32 studies were included for data extraction and analysis. The review was updated as part of the 2018 SCCM clinical practice guidelines, to which 20 other studies were added.4  In the present updated, comprehensive review, 3 new behavioral pain assessment tools developed for other patient populations and tested for their use in critically ill adults (namely, the Escala de Conductas Indicadoras de Dolor [ESCID] Behavioral Indicators for Pain Scale, Multidimensional Objective Pain Assessment Tool [MOPAT], and Nociception Coma Scale-Revised [NCS-R]) and 54 additional studies were included, for a total of 106 articles in which scale development, validation, feasibility, implementation into practice, and/or impact on patient outcomes were described. Items and interpretation of scoring of all behavioral pain assessments tools are described in Table 1, and a summary of all validation and implementation studies is given in Appendixes A and B (available online at www.aacnacconline.org). Results are presented for 9 behavioral pain assessment tools developed for noncommunicative, critically ill adults and 4 such tools developed for other nonverbal populations but tested for their use in noncommunicative, critically ill adults.

Psychometric scoring for each tool is presented in Table 2. Four tools obtained very good weighted scores of greater than 15: the Behavioral Pain Scale (BPS), Behavioral Pain Scale-Nonintubated (BPS-NI), Critical-Care Pain Observation Tool (CPOT), and initial Nonverbal Pain Scale (NVPS-I) with moderate to high evidence. Good weighted scores (ie, scores of 12-14.9) were found for the revised NVPS (NVPS-R; moderate evidence), the MOPAT and the NCS-R-Intubated (NCS-R-I) with low evidence. The Behavior Pain Assessment Tool (BPAT) obtained a high acceptable weighted score (10-11.9) with moderate evidence. Unacceptable weighted scores (ie, those <10) were obtained for all other tools with low or very low evidence.

Behavioral Pain Assessment Tools Developed for Noncommunicative, Critically Ill Adults

Behavior Observation Tool

The psychometric score of the Behavior Observation Tool (BOT) was unchanged, because no new study was retrieved for this updated review. The BOT is a checklist of 38 behaviors clustered into facial responses, verbal responses, and body movements,11  which were inspired by items from previous pain observation tools (namely, the Children’s Hospital of Eastern Ontario Pain Scale12  and the Pain Assessment Intervention Notation [PAIN] Scale13 ). Its content was evaluated by an expert committee for completeness and usability, and the tool was pretested with 10 critically ill adults. This tool was developed for the Thunder Project II and was tested in 5957 critically ill adults from 169 hospitals mainly located in the United States, with a few from Canada, United Kingdom, and Australia.11 

Most patients were admitted to the ICU for a medical (46.5%) or surgical (38%) diagnosis and were able to self-report their pain. Discriminative validation was supported with increases in the frequency of almost all behaviors during standard care procedures. According to criterion validation, positive correlations were found between self-reported procedural pain intensity and the number of facial (r = 0.41), verbal (r = 0.49), and body movement responses (r = 0.37). Patients with procedural pain were 2.8, 4.1, and 10.3 times more likely to have increased facial responses, body movements, and verbal responses, respectively. These findings support the construct validity of the BOT in a large sample of critically ill patients who could communicate.

A modified version of the BOT was tested during a nociceptive procedure (ie, endotracheal suctioning or turning) and gentle touch by Li et al14  in 48 adults who received mechanical ventilatory support and sedation during cardiac surgery. Some behaviors were observed only during the nociceptive procedure, such as grimacing (17%) and random movement of extremities (21%). Additional validation of the BOT is required in noncommunicative patients in the ICU, and reliability needs to be examined.

Behavior Pain Assessment Tool

The BPAT is a shorter version of 8 items of the BOT developed for the Thunder Project II.11  It has been published and was analyzed as part of the 2018 SCCM clinical practice guidelines.4  Initially available in English, the BPAT was translated into 11 different languages (Czech, Dutch, French, German, Greek, Spanish, Italian, Polish, Portuguese, Danish, and Finnish) using a forward method. A teaching video was created for the research teams involved in the validation of the BPAT. The BPAT was validated in 3851 patients from 192 ICU settings in 28 countries.15  Interrater reliability was supported by good to excellent (> 0.60) g coefficients for most behavioral items, which were assessed at rest and during procedures by 2 raters from the ICU care team (eg, nurses, physicians, respiratory therapists, physiotherapists). A few moderate g coefficients were found for muscle rigidity (0.43), wince (0.50), and clenched fists (0.51) at rest. Discriminative validation was demonstrated, with behaviors more likely to be present during procedures than at rest in patients in the ICU who were able or not able to self-report. Regarding criterion validation, moderate correlations were found during procedures between the BPAT scores and the 0 to 10 pain intensity (r = 0.54) and pain distress (r = 0.49) scores. A BPAT cut-point score greater than 3 could adequately classify 75% of patients with or without severe pain with good sensitivity (62%) and specificity (75%). An additional item related to compliance with mechanical ventilatory support would enhance its applicability in patients in the ICU who were receiving such support. The BPAT’s feasibility and implementation in clinical practice need to be studied.

Behavioral Pain Scale and Behavioral Pain Scale–Nonintubated

A total of 39 published ICU studies using the BPS and/or BPS-NI were selected, including 19 new articles published since the 2018 updated SCCM practice guidelines.4  The psychometric scores of both the BPS and the BPS-NI were increased by more than 2 points with cumulative evidence. A training poster for the BPS/BPS-NI16  is available on the SCCM’s ICU Liberation webpage (https://www.sccm.org/ICULiberation/Resources/Behavioral-Pain-Scale-Training -Poster). Studies were conducted in 17 countries. The original versions of the BPS17  and BPS-NI16  were developed in French, and both tools were translated into English using a forward method. The BPS was translated in 10 other languages: Brazilian,18  Brazilian Portu-guese,19  Dutch,20  Italian,21  Chinese,22  Mandarin,23  Norwegian,24  Polish,25  Swedish,26  and Finnish.27  The BPS-NI was translated in 4 other languages: Chinese,28  Italian,21  Norwegian,24  and Swedish.26  All used a forward-backward translation method except for the Dutch version of the BPS,20  for which a forward method was used. The Swedish versions of the BPS and BPS-NI26  were translated following the 10-step process established by the International Society for Pharmacoeconomics and Outcomes.29  The Chinese versions of the BPS and BPS-NI22,28  and the Polish version of the BPS25  were translated using a rigorous, multiple-step process with forward-backward translation, expert committee, and pretest.

The BPS was validated in 25 studies involving 1791 patients in the ICU who had various medical, surgical, and/or trauma diagnoses,17,20,22,23,25,3041  and in studies that included specific ICU groups such as patients with brain injury18,4247  and patients who underwent cardiac surgery.48  The BPS-NI was validated in 2 studies16,21  with 75 patients in medical and surgical ICUs, including 30 with delirium.16  Both tools were simultaneously validated in 6 studies with 658 patients in medical and surgical ICUs.19,24,26,28,48,49 

Internal consistency was examined in 14 studies for the BPS,18,20,23,25,3133,37,4043,47,50  2 studies for the BPS-NI,16,21  and 3 studies for both tools.28,48,49  Good (> 0.70) or acceptable (0.50-0.70) Cronbach α values were found in most studies, and a low value (< 0.20) was only found in patients at rest preprocedure for the Polish version of the BPS.25  However, in 9 studies (69%),16,23,3133,37,42,48,49  Cronbach αs were calculated using dependent data (ie, repeated observations within subjects), which may inflate the coefficient values.

Interrater reliability was tested in 18 studies for the BPS,17,18,20,22,25,3033,35,37,38,4042,46,47,50  both studies for the BPS-NI,16,21  and all 6 studies for both tools.19,24,26,28,48,49  Weighted κ and/or intra-class correlation coefficients (ICCs) greater than 0.60 were reported in most studies, and a lower value (0.36) was obtained at rest for the Norwegian version of the BPS-NI.24  Low percentages of agreement (36%-46%) were also reported during postprocedure assessments for the BPS.40  Such findings may be obtained when raters do not focus on the same behaviors during the observation period. Not enough practice with the tool may also explain lower results. Nurses, physicians, and other health professionals were involved as raters in most studies except that of Liu et al,49  in which raters were the investigator and a research assistant.

Regarding validation strategies, discriminative validation comparing painful and non-painful conditions or standard care procedures (Appendix A) was examined in most validation studies (n = 29 of 33; 88%). Significant increases in BPS/BPS-NI scores during painful procedures were found compared with rest and/or nonpainful procedures in all studies.1622,2426,28,3133,3540,4250  The 2 most common painful procedures were endotracheal/tracheal suctioning (n = 19 studies) and turning or repositioning/changing position (n = 16 studies). Klein et al19  performed a standardized stimulation by pressure algometry for the validation of the Brazilian Portuguese version of the BPS and BPS-NI. Nonpainful procedures were performed in 17 studies, and the 3 most commonly used were eye cleaning (n = 6 studies),18,40,43,4547  noninvasive blood pressure (n = 3 studies),25,35,49  and arterial or central catheter dressing change (n = 3 studies).16,17,31 

Criterion validation was tested in 7 studies for the BPS,22,31,34,35,38,39,42  in 1 study for the BPS-NI,21  and in 1 study for both tools.28  Moderate to high correlations (range, 0.56-0.89) were obtained between BPS/BPS-NI and numeric rating scores (NRS) ranging from 0 to 10 or descriptive scores ranging from 0 to 4.21,28,31,38,39  Lower correlations (< 0.40) were found between BPS and NRS ranging from 0 to 10 when patients were at rest.34,38  A cut-point score was only explored for the English, French, and Chinese versions of the BPS. Receiver-operating curve (ROC) analysis showed that the BPS cut-point ranged from 5 to 6.5, with better classification during painful procedures such as nursing care and endotracheal suctioning (area under the curve [AUC] range, 0.73-0.83)22,35,39  compared with classification at rest (AUC range, 0.60-0.73).34,39  Sensitivity varied from 52% to 90% and specificity from 46% to 92%. Self-reports of pain at rest and during procedures were combined in some studies for the calculation of correlation39  or ROC analysis.35  In the Bernard et al study,42  BPS scores during painful procedures (eg, turning, endotracheal suctioning) were used as the reference standard compared to preprocedure BPS scores in 50 patients in the ICU who had brain injury. Correlated ROC curves were compared and the authors reported an AUC of 0.96 for a BPS cut-point score of 4. The threshold was lower in this sample of patients with brain injury in the ICU than the thresholds obtained when the patient’s self-report of pain was used as the gold standard for this type of analysis. To our knowledge, cut-point scores of other language versions of the BPS and BPS-NI remain to be studied.

The feasibility and clinical relevance of the use of the BPS were evaluated in 4 studies.17,42,48,51  The BPS was evaluated as easy to use or to learn, precise or accurate, and useful by the majority of clinicians (most were nurses). In the initial validation study of the BPS, few evaluators (25%) expressed some concerns regarding its complexity.17  Five implementation studies with results reported in 6 papers in 4 countries (Australia, France, Germany, and Norway) were identified: 3 for the BPS only (French and English versions)5153  and 3 for the BPS and BPS-NI (French and Norwegian versions).5456  Nurses’ adherence to use of the Norwegian version of the BPS/BPS-NI as part of a pain management algorithm was high (75%).55  An increase in the documentation of pain assessments52,56  and decrease in the incidence of severe pain and adverse events51,54  were described. A reduction in mechanical ventilation duration and/or ICU length of stay was found in 2 studies,51,56  but no change was reported in 1 study.53  Implementation of pain and delirium monitoring was associated with a decrease in mortality rate in 1 study.52  Changes in the ordering and/or use of analgesics and sedatives were also described.51,54,56 

Critical-Care Pain Observation Tool

A total of 59 CPOT studies conducted in the ICU setting were included in this review, 35 of which were new articles published after the previous review. The CPOT reached the maximum psychometric score based on cumulative and new evidence. Studies were conducted in 21 countries. The CPOT was initially developed in French57,58  (using various sources, including literature, chart reviews, and consultation with critical care clinicians) and a content validation process,59  and it was translated into English using a forward-backward method.60  The directions for the CPOT use are available in French61  and English62  and a training video was developed by Kaiser Permanente Northern California Nursing Research, which can be found at https://kpnursing.org/professionaldevelopment/index.html as well as on the SCCM’s ICU Liberation webpage (http://www.sccm.org/ICULiberation/Resources/Critical-Care-Pain-Observation-Tool-How-to-Use-it). It is now available in 17 other languages (Appendix A). Most used a forward-backward translation method (ie, the Danish,63  Dutch,64  German,65  Greek,66  Japanese,67  Korean,68  Per-sian,69  Polish,70  Mandarin,23  and Turkish71  versions of the CPOT). The Spanish version72  and another Dutch20  version of the CPOT were translated using a forward method. The translation method of the Italian version of the CPOT was not specified by the authors.73  The Finnish, the Norwegian, and the Swedish versions of the CPOT27,74,75  were translated following the International Society for Pharmacoeconomics and Outcomes 10-step process.29  The Brazilian Portuguese19  and Chinese76  versions were translated using a similar rigorous process with forward-backward translation, cognitive interviewing, expert committee, and pretest. The language version of the CPOT was not specified in 10 studies conducted in countries where English is not the primary language.

In addition to its content validation,59  the CPOT was validated in 47 studies selected for this review, with a total of 3966 patients in the ICU who had various diagnoses and were or were not able to self-report pain. Most studies included patients with medical and/or surgical diagnoses (n = 31 studies)19,20,23,3336,39,48,49,60,61,6365,6870,71,7485 ; some of these studies also included trauma patients (n = 7 studies)60,61,68,69,7981 ; others specifically included patients who had undergone cardiac surgery (n = 9 studies; n = 730 patients; 18%)50,58,71,8691  or who had brain injury (n = 7 studies; n = 690 patients; 17%).44,9297 

Internal consistency was examined in 17 studies.20,23,33,4850,6365,69,71,75,76,78,82,84,96  Good (> 0.70) and acceptable (0.50-0.70) Cronbach α values were found in most studies, and a very low value (0.31) was only found at rest after a procedure, with the Swedish version of the CPOT.75  However, in 7 studies,23,33,49,64,78,82,84  Cronbach α values were calculated using dependent data (ie, repeated observations).

Interrater reliability was reported mainly with weighted κ and/or ICC values in 33 studies,19,20,33,35,4850,58,60,6365,6872,7479,82,85,86,89,9193,9597  and values greater than 0.60 were obtained in 30 of these studies. Clinical staff (mainly nurses) were involved as raters in 22 studies, whereas in other studies, only the investigators and/or research staff were raters. Low values (< 0.40) were reported mainly at rest.86,89  Lower ICCs were found both at rest (0.38) and during turning (0.56) among 4 nurses who used the Dutch version of the CPOT.64  These nurses received a 90-minute standardized training, but there was no information regarding the evaluation of their competence using the Dutch version of the CPOT before data collection. When interrater reliability is not satisfactory, intrarater reliability is a useful strategy to identify low raters.98,99  However, it is only possible to do with videos so raters can view them at a later date (a minimum 1-month interval is recommended to avoid raters recalling their scoring) and perform their scoring of the same patient and under the same conditions. Intrarater reliability was examined in 2 studies86,92  and ICC values greater than 0.80 were obtained for each rater except for 1 value of 0.54 for assessment at rest by 1 rater.86 

Discriminative validation was examined in most of validation studies (n = 43 of 47; 91%). Significant increases in CPOT scores during painful procedures were found in all these studies compared with rest and nonpainful procedures.19,20,33,35,36,39,44,4850,58,60,61,6365,6872,7483,8587,8997  The 2 most common painful procedures were turning or repositioning/changing position (n = 33 studies), and endotracheal/tracheal suctioning (n = 16 studies). Interestingly, oral care procedures (eg, oral suctioning, tooth brushing, swabbing with a sponge toothette) were considered as painful in a study by Dale et al79  but as nonpainful in the studies led by Rijkenberg et al20,50  (no specific description of the procedure was provided). In addition, a standardized stimulation by pressure algometry was performed for validation of the Brazilian Portuguese version of the CPOT.19  Nonpainful procedures were performed in 20 studies; the 2 most commonly used were noninvasive blood pressure using cuff inflation (n = 9 studies)35,49,60,61,76,82,86,92,97  and soft touch (n = 4 studies).65,85,93,95 

Criterion validation was examined in 24 studies using the patient’s self-report of pain presence and/or pain intensity (ie, 0-10 NRS, visual analog scale, or descriptive scale). The CPOT consistently has been associated with self-report of the presence of pain (yes/no). Moderate to high correlations were found with the self-report of pain intensity with higher coefficients (range, 0.42-0.84) during painful procedures.39,58,60,63,77,78,86,93,97  Lower correlations (< 0.40) were found while patients were at rest in a study combining delirious and nondelirious patients,34  in patients with spinal cord or brain injury,96  and in some cardiac surgery samples.88,89  The best CPOT cut-point score that adequately classifies self-reported pain during procedures (AUC range, 72%-91%) varied between 239,65,70,76,79,93,99  and 3.35,60,63  During procedures, sensitivity ranged from 67% to 93% and specificity from 46% to 90%. The CPOT cut-point score was lower at rest and ranged between 159  and 2,34,68,76  with sensitivity ranging from 47% to 81% and specificity from 65% to 97%. Self-reports of pain at rest and during procedures were combined in some studies to establish the best CPOT cut-point score.64,95,96 

The feasibility and/or implementation of CPOT in ICU settings were described in 14 papers. The English, French, and Italian versions of the CPOT were implemented in 6 countries (Australia, Canada, Iran, Italy, United Kingdom, and United States) mainly in mixed ICUs (ie, medical/surgical/trauma). The CPOT was rated as feasible and clinically relevant by ICU nurses.48,61,62,97,101,102  Training in the use of the CPOT allowed nurses to improve their diagnosis of pain in patients with low levels of consciousness.103  However, ICU nurses highlighted that the training did not improve communication of pain assessment findings with physicians who were not familiar with the tool.102  Documentation of pain assessments significantly increased after implementation of the CPOT, reaching or exceeding the minimum frequency interval (ie, every 2-3 hours).104108  Implementation of CPOT and standardized protocols also led to a reduction in sedative use104,105,108,109  and led to appropriate use of opioids based on regular assessments to evaluate effectiveness of analgesia.104107,110  Positive impacts on patient outcomes also were reported, such as fewer complications,110  shorter time requiring mechanical ventilatory support,109  and a low recollection of severe pain by patients surviving their stay in the ICU.104 

Behavioral Indicators of Pain Scale

To our knowledge, the ESCID was analyzed for the first time in the present review. It was developed in Spanish by Latorre-Marco et al111  and is an adaptation of the Campbell’s scale,112  which was suggested by the Analgesia and Sedation Work Group of the Spanish Society of Intensive Care Medicine and Coronary Units.113  As part of the adaptation process, the authors performed content validation with 13 expert clinicians (9 nurses and 4 physicians), and a content validity index was calculated for each item. The lowest content validity index was determined for the item “compliance with ventilator” and it was modified to improve its clarity and meaning. The scale is also available in English,114  but the translation method was not specified and the English version does not appear to have been validated yet.

The ESCID was validated with a total of 356 patients in the ICU setting in Spain: 232 with mixed diagnoses111,114  and 124 patients with trauma.115  Regarding internal consistency, Cronbach α values from 0.69 to 0.85 were obtained for the ESICD total score or when calculated with 1 item deleted, but repeated observations were used in the calculation of these coefficients.111,114  A Cronbach α of 0.63 was also reported in patients who had a Richmond Agitation-Sedation Score (RASS) of −5,114  which appears irrelevant because behaviors are not observable in unresponsive patients. Interrater reliability was examined with raters from the research team and trained nurses in 2 studies114,115  and by members of the research team only in the initial validation study.111  The g coefficients using an ESCID cut-point score of greater than 3 ranged from 0.66 to 1 at all assessment times (ie, before, during, and after procedures) and at the 3 data collection days in the Lopez-Lopez et al study.115  Analyses of variance were not significant when comparing the ESCID mean scores of raters in the Latorre-Marco et al studies.111,114  The authors mentioned checking for intrarater reliability by comparing ESCID scores between procedures for each rater. However, such a procedure is not appropriate, because raters should have observed the patient during the same procedure twice. Considering that acute pain can vary over time, intrarater reliability through the use of videos appears the most appropriate method to allow raters to score the patient again under the same condition but at a later time.

Discriminative validation was examined in all 3 studies. Consistent findings were obtained with higher scores (ie, increases by > 2 points) during painful procedures (eg, turning/repositioning, mobilization, tracheal suctioning) when compared with preprocedure rest and a nonpainful procedure (ie, gentle rub of a gauze cloth on intact skin).111,114,115  Criterion validation remains to be tested and is a useful strategy to establish a cut-point score for clinical use. Feasibility and implementation of the ESCID in clinical practice also need to be studied.

Nonverbal Pain Assessment Tool

The Nonverbal Pain Assessment Tool (NPAT) is available in English and was validated in 220 medical and surgical patients from 4 adult ICUs.116  No new article was retrieved for this tool when compiling the present review, and the NPAT was described in our previous review.7  Briefly, internal consistency (α = 0.82) was high, and interrater reliability was moderate to high (r = 0.52-0.88) between 2 nurses’ rating scores (5 teams of 2 nurses participated). Low correlation coefficients (ie, 0.21-0.31) were found between NPAT scores and patients’ self-reports of pain intensity on a scale of 0 to 10. No information about the conditions of pain assessments (eg, at rest, during care procedures) was provided. These findings supported the reliability but not the validity of the NPAT use, and additional validation of the tool would be necessary.

Nonverbal Pain Scale—Initial and Revised Versions

Of the 12 studies using NVPS-I or NVPS-R included in the present review, 5 were published after the previous review. Ten were validation studies with a total of 778 patients in the ICU, and 2 were implementation studies. The psychometric scores of both versions of the NVPS were increased by more than 3 points with new evidence. Two versions of the NVPS exist: the NVPS-I and the NVPS-R. The initial version of the NVPS was developed in English and includes 3 behavioral and 2 physiological indicators; its content was evaluated by critical care experts.117  The NVPS-R, in which the skin physiologic indicator was modified for a respiratory item (ie, respiratory rate, oxygen saturation, compliance with ventilator), is also available.118  In addition to the English version, the NVPS-I was translated into Polish using a forward method and a content validation process with ICU nurses.25  The NVPS-R was translated into Finnish27  using the International Society for Pharmacoeconomics and Outcomes 10-step process, and into Turkish using a forward-backward method and a content validation process with 9 specialists119 ; an overall content validity index of 1.00 was reported for the Turkish version. Both versions of the tool were translated for a study conducted in Iran, but neither the language nor the translation method was specified by the authors.120 

The NVPS-I was validated in 3 studies with 237 patients with medical, surgical, trauma, neurologic, or burn diagnoses.25,38,117  The NVPS-R was validated in 5 studies with 417 patients with medical, surgical, trauma, and neurologic diagnoses.33,37,48,97,119  Both versions of the tool were simultaneously validated in 2 studies with 124 patients with various diagnoses.118,120 

Internal consistency was examined in 8 studies and results were variable for both the NVPS-I and the NVPS-R: Cronbach α coefficients ranged from 0.36 to 0.72 at different times.25,118  The highest values of 0.62 and 0.78 were obtained during the painful procedure for the NVPS-I118  and the Turkish version of the NVPS-R,119  respectively. The value of Cronbach α was improved during the nonpainful and painful procedures when physiologic indicators (eg, vital signs, skin) were removed in the Polish version of the NVPS-I.25  Higher Cronbach α values (0.75-0.86) were obtained when repeated observations were used for the calculation of the coefficients.33,37,48,117,120 

Interrater reliability was tested with raters from the research team and ICU nurses in most studies but not in that of Rahu et al,38  in which only the investigators were involved as raters. Percentages of agreement (> 90%)118  as well as ICC values and g coefficients were high (0.60-0.95) in several studies.33,37,49,120  Lower interrater reliability coefficients were found with the Polish version of the NVPS-I (g = 0.44)25  and for the English version of the NVPS-R, with ICCs in the latter ranging from 0.34 to 0.49 at rest before and after procedures and an ICC of 0.55 during the nonpainful procedure (ie, noninvasive blood pressure).121 

Discriminative validation was tested in most studies (n = 7 of 10; 70%) and was supported by significant increases in the tool’s scores during painful procedures compared with rest and nonpainful procedures.25,37,38,48,97,118,120  The most common painful procedures were endotracheal suctioning and/or turning or repositioning,25,37,38,48,97,118,120  and various nonpainful procedures were used across studies. It is worth mentioning that the skin indicator of the NVPS-I did not increase significantly during endotracheal suctioning.118 

Criterion validation was explored in only one study of the NVPS-I38  and one of the NVPS-R97 ; in the Iranian study, both versions were used.120  In association with the 0 to 10 scale of self-reported pain intensity, a low correlation of 0.31 was found during turning for the English version of the NVPS-R,97  and moderate correlations 0.41 and 0.56 were obtained during physical examination and during endotracheal suctioning for the English version of the NVPS-I, respectively.38  Using 269 self-reports of the presence or absence of pain from 60 patients in the ICU, the authors of the Iranian study reported sensitivity and specificity greater than 95%, with a cut-point score of 1.5 for both the translated versions of the NVPS-I and of the NVPS-R.120 

The implementation of the English version of the NVPS-I in 2 neurological and trauma ICU settings was evaluated in Canada121  and in the United States.122  In the latter study, by Sacco et al,122  the NVPS-I was implemented as part of an analgesia and sedation guideline. There was an increase in the documentation of pain assessments, and patients reported an overall decrease in their pain after implementation of the tool.121  A decrease in the average number of sedation days and in analgesic treatment duration was found in that implementation study.122  The tool or guideline was rated as easy to use by ICU nurses in both studies. Although the ICU nurses’ satisfaction with the guideline was positive in the Sacco et al study,122  nurses in the Topolovec-Vranic et al study121  were less likely to agree that the NVPS-I would ease the assessment of their patients’ pain, make them confident to request fewer or more analgesics, or improve their pain management practice. The feasibility and clinical relevance of the English version of the NVPS-R was evaluated by 10 ICU nurses in 1 validation study in Canada.97  Although the NVPS-R was rated as easy to use by nurses, its clinical relevance was only supported by 20% of them. Physicians’ practices with regard to pain management and inconsistent pain assessment and management practices of the staff were mentioned as major barriers by ICU nurses.97 

Pain Assessment Intervention Notation

The PAIN was analyzed in our previous review,7  and no new article was retrieved for the present update. The PAIN algorithm is a tool developed by a team of critical care nurses and pain experts and intended to combine pain assessments with subsequent pain interventions.13  Clinicians are directed to first use a checklist of 4 behaviors and 5 physiologic signs to determine the presence or absence of pain. The nurse is then prompted to elicit the patient’s degree of pain intensity on a 0 to 10 NRS. The management steps of the algorithm involve assessing for potential problems influencing opioid administration and making an analgesic treatment decision. The PAIN tool was tested with 31 recently extubated, self-reporting adults in the ICU after surgery.13  The number of behavioral and physiological indicators observed was significantly associated with the nurses’ proxy ratings of 0 to 10 pain intensity on an NRS (r = 0.17 to 0.77). However, the relationship between the patient’s self-report of pain intensity and behavioral and physiological indicators was not calculated; therefore, criterion validation was not described in this sample. Nine of 11 nurse evaluators of 31 patients deemed the instrument to be helpful, whereas a few nurses (n = 4) found the PAIN tool to be too complex.123  Although the PAIN tool was tested in a small sample of patients in the ICU who were able to communicate, most psychometric properties were not examined. The PAIN tool is based on the nurse’s clinical judgment and appears to be more of a pain-management educational tool than a single pain scale per se.

Behavioral Pain Assessment Tools Developed for Non-ICU Populations

Face, Legs, Activity, Cry, Consolability

The psychometric analysis of the Face, Legs, Activity, Cry, Consolability (FLACC) was described in our previous review,7  and no new article was retrieved for the present update. The tool was initially developed for children with cognitive impairment,124  and its use was validated in a sample of 29 adults and 8 pediatric patients from medical and surgical ICUs by Voepel-Lewis et al.125  A Cronbach α coefficient of 0.88 was obtained using dependent data (n = 73 observations from 37 patients), which was improved to 0.93 by removing the “cry” item from the scale. Such a finding may indicate that the cry item is not relevant to adults in the ICU. Interrater reliability was demonstrated with an ICC of 0.98 for the FLACC scores between 2 nurse raters who completed 60 observations in 29 adult patients in the ICU. Significant reductions in FLACC scores (P < .001) comparing before and after analgesic administration or painful and non-painful procedures supported discriminative validation. Additional research is necessary to adapt the content of the FLACC for the ICU adult population and to establish the reliability and validity of its use in noncommunicative, critically ill adults.

Multidimensional Objective Pain Assessment Tool

The Multidimensional Objective Pain Assessment Tool (MOPAT) was newly analyzed in this review. The tool includes 4 behavioral and 4 physiologic indicators. Its content development was based on the ratings of pain descriptors by 20 nurses according to their own experience of assessing pain in non-communicative and cognitively impaired patients in hospice.126  The validity, reliability, and clinical usefulness of the MOPAT was tested in noncommunicative hospice patients.126  The blood pressure indicator was reported as having a negligible effect.126 

The MOPAT (including the blood pressure indicator) was evaluated in 27 patients in the medical ICU who were unable to self-report pain.127  Internal consistency of MOPAT total scores was moderate during and after painful procedures, with Cronbach α values of 0.68 and 0.72, respectively. Cronbach α values were higher for the behavioral component (≥ 0.80) compared with the physiologic component during and after the painful procedures (0.37 and 0.57, respectively). Interrater agreement among 3 raters (from 2 investigators and 21 nurses) was 68% and 83% for the behavioral component, 80% and 79% for the physiologic component (g coefficients not provided) at the 2 times. Discriminative validation was demonstrated for behavioral and physiologic indicators, as well as for the MOPAT total score—all showing significantly higher scores during versus after a painful procedure (eg, turning, suctioning). The physiologic indicators had a very small effect on the variation of the total score. The Clinical Utility Questionnaire was used to evaluate the nurses’ perceptions regarding the usefulness of the MOPAT. The tool was considered easy to use and helpful in determining the presence of pain in a noncommunicative patient by more than 93% of the responders, but 20% were undecided about whether the tool assisted them in communicating to others about a patient’s pain.127 

The MOPAT was validated in a small ICU sample but was evaluated as feasible and clinically useful by ICU nurses.127  Overall, the behavioral indicators appear to be the most helpful, whereas the physiologic indicators require additional study. The item related to “patient sounds” is not applicable to patients receiving mechanical ventilatory support and would require an alternative item for this ICU clientele. Implementation studies in clinical practice are needed.

Nociception Coma Scale–Revised

The NCS-R128  is a short version (with the visual response removed) of the 4-item NCS developed for nociception and pain assessment in patients with brain injury with disorders of consciousness.129  The initial version of the NCS was developed on the basis of an extensive literature review and pilot data, and it was tested in 48 patients from various settings (eg, acute care, neurology, neurorehabilitation, nursing homes) but not in the ICU. The NCS-R was newly analyzed in the present review. It was tested in a total of 60 acute care patients with brain injury in 2 studies by Chatelle et al.128,130  In their 2012 study,128  21 patients with acute brain injury (including patients in the ICU) were part of a larger sample with chronic patients who were 1 month to 6 years after injury (n = 64). Patients were assessed at baseline (rest), with a non-nociceptive stimulation (taps on the patient’s shoulders), and with a nociceptive stimulation (pressure on the nail bed measured in Newton-meters). In this study,128  Chatelle et al initially used the NCS, with which they obtained poor validity results. They decided to remove the visual response item, which led to improved findings. The NCS-R could discriminate among baseline, the non-nociceptive, and the nociceptive stimulations. The researchers used ROC analysis to compare curves between the non-nociceptive and the nociceptive stimulations.128  A cut-point score of 4 with a sensitivity of 73% and a specificity of 97% was found. The NCS-R cut-point score varied according to the patient’s level of consciousness and was lower (score of 3) for those in a vegetative state. In the 2016 study of Chatelle et al,130  discriminative validation of the NCS-R was supported with significant lower scores after administration of an analgesic compared with before administration in 39 patients with brain injury who were in the ICU and neurology units. Chatelle et al used an NCS-R cut-point score of 4 as a criterion to confirm the presence of pain before the administration of an analgesic. Neither internal consistency nor interrater reliability were examined in these studies.128,130  Of note, the score of 3 for the facial expression item (the cry item) is greater than that given for grimace (score of 2). It remains unclear from the scoring description if facial contraction should be present in addition to cry to obtain the highest score on this item.129  Also, the item “verbal response” would not be applicable to patients receiving mechanical ventilatory support.

To make the NCS-R applicable to patients in the ICU receiving mechanical ventilatory support, Bernard et al42  replaced the verbal response item of the NCS-R with the compliance with ventilator item of the BPS and called it the NCS-R-I (I for intubated). They tested this version in 50 patients in the ICU with brain injury who were intubated. A Cronbach α of 0.69 was found for internal consistency, and interrater reliability was supported with a weighted g of 0.84 between the investigator and nurse raters. Significant increases in the NCS-R-I scores were found during both the non-nociceptive (ie, assessment of the sedation level using the RASS procedure) and the nociceptive procedures (ie, tracheal suctioning and turning), but the increase was greater during the nociceptive procedures. Using the nociceptive procedures as the reference criterion and the non-nociceptive procedure as a comparator, correlated ROC curves resulted in an AUC of 0.97 for an NCS-R-I cut-point score of 2. ICU nurses (n = 15 of 21; 71% response rate) evaluated the NCS-R-I as precise (93%), useful (100%), and easy to learn (80%).

Pain Assessment in Advanced Dementia Tool

The Pain Assessment in Advanced Dementia (PAINAD) tool was developed to assess pain in individuals with advanced dementia.10  It was validated by Paulson-Conger et al84  for use in 100 adult patients in 4 medical ICUs (cardiac, medical, surgical, and neurologic) of a level I trauma center in the United States. The PAINAD tool was analyzed as part of the 2018 SCCM clinical practice guidelines,4  and 1 new validation study with 50 patients who were admitted to the ICU for spinal cord or brain injury was added in this review,96  which led to a 3-point increase in its psychometric score. In the study by Paulson-Conger et al,84  patients were assessed only once, while at rest, by the principal investigator or trained data collectors (1 per critical care unit). Internal consistency was supported with a Cronbach α of 0.80. Correlation between PAINAD and CPOT scores was 0.86. Such a high correlation between the tools’ scores is not surprising—they include similar items. Comparing 2 tools measuring the same construct (in this case, pain behaviors) refers to convergent validation but, ideally, it should be done by selecting 2 different assessment methods and not similar tools.99  In the study by Sulla et al,96  a correlation of 0.67 was found between the PAINAD and self-reported NRS scores and almost a perfect classification ability (AUC = 0.98) using repeated observations. The use of the PAINAD tool in critically ill adults cannot be supported, because some validation strategies remain to be examined (ie, interrater reliability) or replicated (ie, criterion and discriminative validation), and some tests (correlation and ROC curve analysis) were performed using methods not considering dependent data. In addition, the PAINAD tool’s content should be adapted for use with patients receiving mechanical ventilatory support, for whom the item “vocalization” is not applicable.

Discussion

Validation studies of behavioral pain assessment tools in critically ill adults have increased substantially in recent years. In fact, half of all studies included in this review were published in the last 3 years. Considering these additional studies, higher psychometric scores were obtained for many tools, including the BPS, BPS-NI, CPOT, NVPS-I, NVPS-R, and PAINAD, when compared with those of the 2018 SCCM clinical practice guidelines.4  The BPS, BPS-NI, and the CPOT remain the behavioral pain assessment tools with the strongest psychometric properties, as stated in the 2018 SCCM guidelines.4  These tools underwent external validation in several countries (not including Canada and France, where they were originally developed) and have been translated into several different languages. Comparison of the BPS and the CPOT yielded similar psychometric results.19,20,23,33,35,36,39,4850  The BPAT is the tool that was validated in the most countries (n = 28) and is available in multiple languages. Good psychometric findings have been reported for the initial and revised versions of the NVPS for their use in the ICU; however, the physiologic indicators did not perform as well as the behavioral indicators. Similar findings were obtained using the MOPAT, with the behavioral indicators performing better compared with physiologic indicators. Tools developed for pediatric and non-ICU populations would require some adaptation to be applicable to the adult ICU population. This strengthens the fact that use of a tool is valid for a specific population and in a given context,99  and that use of the tool in a different patient group or context of care is contingent on its additional validation in these new circumstances.

In many studies, a lack of attention to basic assumptions in statistics was noted. The most frequent was the calculation of Cronbach α or correlation coefficients on the basis of the number of observations (eg, 300 observations) within subjects (eg, 30 patients) assessed at different times, which may inflate the values of these coefficients. A more accurate approach is to calculate and report these coefficients using the sample of patients at each assessment time (eg, before, during, and after procedures). A Cronbach α is based on shared variance among items, which may be low when calculated at rest because the tools’ scores tend to be low. Cronbach α also is influenced by the number of items (ie, a greater number of items increases the coefficient value).131  Therefore, we may question the relevance of this statistic for the validation of behavioral pain assessment tools.

Confusion regarding validation strategies was also noted. In several studies, authors reported a correlation coefficient between the scores of 2 scales with similar items (eg, BPS and CPOT) and described this strategy as convergent or a criterion validation. Convergent validation refers to comparing 2 different assessment methods measuring the same concept (in this review, the concept is related to pain behaviors). Using similar assessment methods (eg, 2 behavioral pain assessment tools), therefore, is not the most useful approach, because the correlation tends to be high (> 0.80), as reported in studies19,20,23,33,3537,39,4850,85,96,119  included in this review. The use of other pain-related measures such as pain distress15  or pain unpleasantness86  would be more appropriate for convergent validation testing. Criterion validation refers to comparing the tool with the gold standard established in the field.99  The patient’s self-report is the gold standard criterion in the field of pain. Therefore, using 2 behavioral pain assessment tools for criterion validation is not appropriate.

ROC analysis is a common strategy to establish the performance of a test or tool to detect a problem as well as sensitivity and specificity associated with the best cut-point score.132  A gold standard criterion must be present that allows discrimination of patients into 2 groups: 1 with pain and 1 without pain. Because behavioral pain assessment tools are developed for use with noncommunicative patients in the ICU, cut-point scores for these tools were established in most studies using the self-report of communicative patients in the ICU.12,22,34,35,39,60,63,68,70,76,79,93,100  Although it may represent the best available approach according to the pain definition, this strategy may have limitations because the patient’s self-report of pain must be reliable and is influenced by many personal and social factors. Delirium is a common source of unreliable self-report, and screening for delirium was not specified in many studies. Moreover, this approach may be appropriate if we assume that pain behaviors are the same in communicative and noncommunicative patients in the ICU. The level of consciousness and the level of sedation or agitation may influence pain behaviors. The threshold could be lower in patients with a low level of consciousness or a high level of sedation42  or higher in agitated patients.69  Another approach was explored by some research teams who used nociceptive procedures as the reference criterion in comparison with rest before a procedure or used non-nociceptive procedures to assess the ability of behavioral tools to detect pain.42,128  However, the way the authors dealt with dependent data (measurements during both conditions) within the ROC analysis was not specified. Also, not all patients experience pain during nociceptive procedures; therefore, the cut-point score instead would detect a nociceptive procedure (or nociception) than detect pain per se. For consistency in scoring criterion validation, we gave scores to studies in which the patient’s self-report of pain was used as the gold standard. In future research, both criteria (ie, self-report of pain, nociceptive procedures) could be explored in ROC analysis to allow comparison between cut-point scores.

In the 2013 SCCM clinical practice guidelines,8  validation of behavioral pain assessment tools in patients with brain injury in the ICU was identified as an area for additional research. Since then, this patient group was specifically examined in several studies with the BPS,18,4247  the CPOT,44,9297  the NCS-R,128,130  and the NCS-R-I.42  When compared with patients undergoing surgery and those with trauma, those with a brain injury had lower CPOT scores.80  Interestingly, a low effect size was found for the facial expression item of the BPS,47  and a grimace score of 2 and muscle rigidity score of 2 on the CPOT were not frequently observed.93  In addition, specific behaviors such as orbit tightening and eye weeping were described in patients with brain injury in the ICU.133,134  Adaptation of the content of existing scales for such patients could enhance their applicability and validity in this vulnerable group.

Potential sources of bias in conducting this analysis include the selection and weighting of items in the psychometric scoring system that influenced the scores of each scale. Also, psychometric properties are not static and may evolve with further development or adaptation and validation testing of a scale. The use of a different system would lead to different scores for each scale. Psychometric scores also were established for the use in scales for adult patients in the ICU. Therefore, they do not reflect the psychometric performance of their use in other contexts of care.

Implications for Nursing

Nurses play a key role in the assessment and management of pain, and they are advocates for their patients to ensure that pain does not go unnoticed. They are responsible for regularly assessing pain using methods appropriate to a patient’s ability to communicate and to offer adequate treatment based on a multimodal analgesic approach. Behavioral pain scores as part of pain management algorithms3,24,54  may guide nurses in the administration of a low opioid dose when appropriate, and pain reassessment is essential to evaluate the effectiveness of analgesia and adjust the treatment. Nurses can also initiate nonpharmacologic interventions to optimize pain management.4  Family members can be consulted about pain-related behaviors of their loved one and offered the opportunity to contribute to pain management if they feel comfortable doing so.46 

Pain management is a team effort and, to optimize the nurse’s role in pain management, training on any tool’s use should be offered to all members of the multidisciplinary team. Communication about pain assessment and management would be enhanced if addressed during nursing handoffs and multidisciplinary daily rounds, with all attendees understanding the methods used for obtaining crucial information about the patient’s pain. In addition, ICU leaders, such as clinical nurse specialists or educators, are encouraged to conduct routine quality control on the use of pain assessment tools by clinicians so accurate information is used to make treatment decisions.

Future Research Steps

Implementation studies have focused mainly on pain management practice indicators and short-term patient outcomes. Still unexplored, however, is the impact pain management protocols based on the regular use of pain assessment tools have on long-term patient outcomes, such as chronic pain development and quality of life. In addition, behavioral pain assessment tools can only be used in patients able to behaviorally react to stimulation. Therefore, they are minimal or absent in patients who are heavily sedated (eg, RASS of −4) and are not observable in unresponsive patients (eg, Glasgow Coma Scale score of 3; RASS of −5). Behavioral pain assessment tools can also be challenging to use in very agitated patients.68  In such situations, clinicians can consider that a patient might be in pain if a responsive patient in a similar condition would most likely report pain, and alternative assessment methods must be explored. Although vital signs are not valid for pain assessment purposes in the ICU when considered individually,36  innovative technology may have some potential. The use of the Analgesia Nociception Index (which is based on heart rate variability) and the Nociception Level Index (which is based on multiple parameters related to heart rate, heart rate variability, galvanic skin response, and peripheral temperature) have been studied for ICU pain assessment,44,135,136  but more research is needed.

Conclusions

The use of validated behavioral pain assessment tools is crucial for noncommunicative, critically ill patients. The BPS, BPS-NI, and CPOT, which were specifically developed for this population, have shown the strongest psychometric properties with highest evidence; their use is feasible and positively influenced pain management practices and patient outcomes. It is important that all members of the multidisciplinary team be trained to use the behavioral pain assessment tool selected for their ICU setting so they can interpret pain scores and better communicate about pain assessment findings. Assessing pain properly is the key step to improved pain management and better care for this vulnerable ICU population.

REFERENCES

REFERENCES
1
Chanques
G
,
Sebbane
M
,
Barbotte
E
,
Viel
E
,
Eledjam
JJ
,
Jaber
S
.
A prospective study of pain at rest: incidence and characteristics of an unrecognized symptom in surgical and trauma versus medical intensive care unit patients
.
Anesthesiology
.
2007
;
107
(
5
):
858
860
.
2
Puntillo
KA
,
Max
A
,
Timsit
JF
, et al
.
Determinants of procedural pain intensity in the intensive care unit: the Europain study
.
Am J Respir Crit Care Med
.
2014
;
189
(
1
):
39
47
.
3
Gélinas
C
.
Pain assessment in the critically ill adult: recent evidence and new trends
.
Intensive Crit Care Nurs
.
2016
;
34
:
1
11
.
4
Devlin
JW
,
Skrobik
Y
,
Gélinas
C
, et al
.
Clinical practice guidelines for the prevention and management of pain, agitation/sedation, delirium, immobility, and sleep disruption in adult patients in the ICU
.
Crit Care Med
.
2018
;
46
(
9
):
e825
e873
. doi:
5
Clinical Resources Task Force, American Asssociation of Critical-Care Nurses
.
Assessing pain in critically ill adults
.
Crit Care Nurs
.
2018
;
38
(
6
):
e13
e16
.
6
Herr
K
,
Coyne
PJ
,
McCaffery
M
,
Manworren
R
,
Merkel
S
.
Pain assessment in the patient unable to self-report: position statement with clinical practice recommendations
.
Pain Manag Nurs
.
2011
;
12
(
4
):
230
250
.
7
Gélinas
C
,
Puntillo
KA
,
Joffe
AM
,
Barr
J
.
A validated approach to evaluating psychometric properties of pain assessment tools for use in nonverbal critically ill adults
.
Semin Respir Crit Care Med
.
2013
;
34
(
2
):
153
168
.
8
Barr
J
,
Fraser
GL
,
Puntillo
K
, et al
.
Clinical practice guidelines for the management of pain, agitation, and delirium in adult patients in the intensive care unit
.
Crit Care Med
.
2013
;
41
(
1
):
263
306
.
9
Devlin
JW
,
Skrobik
Y
,
Rochwerg
B
, et al
.
Methodologic innovation in creating clinical practice guidelines: insights from the 2018 Society of Critical Care Medicine pain, agitation/sedation, delirium, immobility, and sleep disruption guideline effort
.
Crit Care Med
.
2018
;
46
(
9
):
1457
1463
.
10
Warden
V
,
Hurley
AC
,
Volicer
L
.
Development and psychometric evaluation of the Pain Assessment in Advanced Dementia (PAINAD) scale
.
J Am Med Dir Assoc
.
2003
;
4
(
1
):
9
15
.
11
Puntillo
KA
,
Morris
AB
,
Thompson
CL
,
Stanik-Hutt
J
,
White
CA
,
Wild
LR
.
Pain behaviors observed during six common procedures: results from Thunder Project II
.
Crit Care Med
.
2004
;
32
(
2
):
421
427
.
12
McGrath
P
,
Johnson
G
,
Goodman
J
,
Dunn
J
,
Chapman
J
.
CHEOPS: a behavioral scale for rating postoperative pain in children
. In:
Field
HL
,
Dubner
R
,
Cervero
R
, eds.
Advances for Pain Research and Therapy
. Vol
9
.
New York, NY
:
Raven
;
1985
:
395
402
.
13
Puntillo
KA
,
Miaskowski
C
,
Kehrle
K
,
Stannard
D
,
Glee-son
S
,
Nye
P
.
Relationship between behavioral and physiological indicators of pain, critical care patients’ self-reports of pain, and opioid administration
.
Crit Care Med
.
1997
;
25
(
7
):
1159
1166
.
14
Li
D
,
Miaskowski
C
,
Burkhardt
D
,
Puntillo
K
.
Evaluations of physiologic reactivity and reflexive behaviors during noxious procedures in sedated critically ill patients
.
J Crit Care
.
2009
;
24
(
3
):
472.e9
472.e13
. doi:
15
Gélinas
C
,
Puntillo
KA
,
Levin
P
,
Azoulay
E
.
The behavior pain assessment tool for critically ill adults: a validation study in 28 countries
.
Pain
.
2017
;
158
(
5
):
811
821
.
16
Chanques
G
,
Payen
JF
,
Mercier
G
, et al
.
Assessing pain in non-intubated critically ill patients unable to self report: an adaptation of the Behavioral Pain Scale
.
Intensive Care Med
.
2009
;
35
(
12
):
2060
2067
.
17
Payen
JF
,
Bru
O
,
Bosson
JL
, et al
.
Assessing pain in critically ill sedated patients by using a behavioral pain scale
.
Crit Care Med
.
2001
;
29
(
12
):
2258
2263
.
18
Azevedo-Santos
IF
,
Alves
IG
,
Badaue-Passos
D
,
Santana-Filho
VJ
,
DeSantana
JM
.
Psychometric analysis of behavioral pain scale Brazilian version in sedated and mechanically ventilated adult patients: a preliminary study
.
Pain Pract
.
2016
;
16
(
4
):
451
458
.
19
Klein
C
,
Caumo
W
,
Gélinas
C
, et al
.
Validation of two pain assessment tools using a standardized nociceptive stimulation in critically ill adults
.
J Pain Symptom Manage
.
2018
;
56
(
4
):
594
601
.
20
Rijkenberg
S
,
Stilma
W
,
Endeman
H
,
Bosman
RJ
,
Oudemans-van Straaten
HM
.
Pain measurement in mechanically ventilated critically ill patients: Behavioral Pain Scale versus Critical-Care Pain Observation Tool
.
J Crit Care
.
2015
;
30
(
1
):
167
172
.
21
Carletti
SAC
,
Rossi
E
,
Destrebecq
A
,
Terzoni
S
,
Bonetti
L
.
Translation and validation in Italian of the behavioral pain scale non-intubated patients (BPS-NI) for pain assessment in non-intubated critically ill patients, unable to self-report [in Italian]
.
Scenario
.
2016
;
33
(
1
):
4
13
.
22
Chen
YY
,
Lai
YH
,
Shun
SC
,
Chi
NH
,
Tsai
PS
,
Liao
YM
.
The Chinese Behavior Pain Scale for critically ill patients: translation and psychometric testing
.
Int J Nurs Stud
.
2011
;
48
(
4
):
438
448
.
23
Hsiung
NH
,
Yang
Y
,
Lee
MS
,
Dalal
K
,
Smith
GD
.
Translation, adaptation, and validation of the behavioral pain scale and the critical-care pain observational tools in Taiwan
.
J Pain Res
.
2016
;
9
:
661
669
.
24
Olsen
BF
,
Rustoen
T
,
Sandvik
L
,
Miaskowski
C
,
Jacobsen
M
,
Valeberg
BT
.
Development of a pain management algorithm for intensive care units
.
Heart Lung
.
2015
;
44
(
6
):
521
527
.
25
Gutysz-Wojnicka
A
,
Ozga
D
,
Mayzner-Zawadzka
E
,
Dyk
D
,
Majewski
M
,
Doboszynska
A
.
Psychometric assessment of physiologic and behavioral pain indicators in Polish versions of the pain assessment scales
.
Pain Manag Nurs
.
2019
;
20
(
3
):
292
301
.
26
Hylén
M
,
Akerman
E
,
Alm-Roijer
C
,
Idvall
E
.
Behavioral Pain Scale - translation, reliability, and validity in a Swedish context
.
Acta Anaesthesiol Scand
.
2016
;
1
(
6
):
821
828
.
27
Pudas-Tahka
SM
,
Axelin
A
,
Aantaa
R
,
Lund
V
,
Salantera
S
.
Translation and cultural adaptation of an objective pain assessment tool for Finnish ICU patients
.
Scand J Caring Sci
.
2014
;
28
(
4
):
885
894
.
28
Chen
J
,
Lu
Q
,
Wu
XY
,
An
YZ
,
Zhan
YC
,
Zhang
HY
.
Reliability and validity of the Chinese version of the behavioral pain scale in intubated and non-intubated critically ill patients: two cross-sectional studies
.
Int J Nurs Stud
.
2016
;
61
:
63
71
.
29
Wild
D
,
Grove
A
,
Martin
M
, et al
.
Principles of good practice for the translation and cultural adaptation process for patient-reported outcomes (PRO) measures: report of the ISPOR Task Force for Translation and Cultural Adaptation
.
Value Health
.
2005
;
8
(
2
):
94
104
.
30
Ahlers
SJ
,
van Gulik
L
,
van der Veen
AM
, et al
.
Comparison of different pain scoring systems in critically ill patients in a general ICU
.
Crit Care
.
2008
;
1
(
1
):
R15
.
31
Ahlers
SJ
,
van der Veen
AM
,
van Dijk
M
,
Tibboel
D
,
Knibbe
CA
.
The use of the Behavioral Pain Scale to assess pain in conscious sedated patients
.
Anesth Analg
.
2010
;
110
(
1
):
127
133
.
32
Aissaoui
Y
,
Zeggwagh
AA
,
Zekraoui
A
,
Abidi
K
,
Abouqal
R
.
Validation of a behavioral pain scale in critically ill, sedated, and mechanically ventilated patients
.
Anesthes Analg
.
2005
;
101
(
5
):
1470
1476
.
33
Al Darwish
ZQ
,
Hamdi
R
,
Fallatah
S
.
Evaluation of pain assessment tools in patients receiving mechanical ventilation
.
AACN Adv Crit Care
.
2016
;
27
(
2
):
162
172
.
34
Bouajram
RH
,
Sebat
CM
,
Love
D
,
Louie
EL
,
Wilson
MD
,
Duby
JJ
.
Comparison of self-reported and behavioral pain assessment tools in critically ill patients
[published online February 1, 2018]
.
J Intensive Care Med
. doi:
35
Cheng
L-H
,
Tsai
Y-F
,
Wang
C-H
,
Tsay
P-K
.
Validation of two Chinese-version pain observation tools in conscious and unconscious critically ill patients
.
Intensive Crit Care Nurs
.
2018
;
44
:
115
122
.
36
Gomarverdi
S
,
Sedighie
L
,
Seifrabiei
MA
,
Nikooseresht
M
.
Comparison of two pain scales: Behavioral Pain Scale and Critical-Care Pain Observation Tool during invasive and noninvasive procedures in intensive care unit-admitted patients
.
Iran J Nurs Midwifery Res
.
2019
;
24
(
2
):
151
155
.
37
Juarez
P
,
Bach
A
,
Baker
M
, et al
.
Comparison of two pain scales for the assessment of pain in the ventilated adult patient
.
Dimens Crit Care Nurs
.
2010
;
29
(
6
):
307
315
.
38
Rahu
MA
,
Grap
MJ
,
Ferguson
P
,
Joseph
P
,
Sherman
S
,
Elswick
RK
.
Validity and sensitivity of 6 pain scales in critically ill, intubated adults
.
Am J Crit Care
.
2015
;
24
(
6
):
514
523
.
39
Severgnini
P
,
Pelosi
P
,
Contino
E
,
Serafinelli
E
,
Novario
R
,
Chiaranda
M
.
Accuracy of Critical Care Pain Observation Tool and Behavioral Pain Scale to assess pain in critically ill conscious and unconscious patients: prospective, observational study
.
J Intensive Care
.
2016
;
4
:
68
.
40
Young
J
,
Siffleet
J
,
Nikoletti
S
,
Shaw
T
.
Use of a Behavioural Pain Scale to assess pain in ventilated, unconscious and/or sedated patients
.
Intensive Crit Care Nurs
.
2006
;
22
(
1
):
32
39
.
41
Navarro-Colom
M
,
Sendra-Lluis
MA
,
Castillo-Masa
AM
,
Robleda
G
.
Intraobserver reliability and internal consistency of the Behavioral Pain Scale in mechanically-ventilated patients [in Spanish]
.
Enferm Intensiva
.
2015
;
26
(
1
):
24
31
.
42
Bernard
C
,
Delmas
V
,
Duflos
C
, et al
.
Assessing pain in critically ill brain-injured patients: a psychometric comparison of 3 pain scales and videopupillometry
[published online June 10, 2018]
.
Pain
. doi:
43
Dehghani
H
,
Tavangar
H
,
Ghandehari
A
.
Validity and reliability of behavioral pain scale in patients with low level of consciousness due to head trauma hospitalized in intensive care unit
.
Arch Trauma Res
.
2014
;
3
(
1
):
e18608
. doi:
44
Jendoubi
A
,
Abbes
A
,
Ghedira
S
,
Houissa
M
.
Pain measurement in mechanically ventilated patients with traumatic brain injury: behavioral pain tools versus analgesia nociception index
.
Indian J Crit Care Med
.
2017
;
21
(
9
):
585
588
.
45
Ribeiro
CJN
,
Bezerra
DS
,
Lima
AGCF
,
Fernandes
CCF
,
Menezes
MGdV
Ribeiro MdCdO
.
Pain during tracheal aspiration in patients with traumatic brain injury undergoing mechanical ventilation
.
Revista Dor
.
2017
;
18
:
332
337
.
46
Ribeiro
CJN
,
Araujo
ACS
,
Brito
SB
, et al
.
Pain assessment of traumatic brain injury victims using the Brazilian version of the Behavioral Pain Scale
.
Rev Bras Ter Intensiva
.
2018
;
30
(
1
):
42
49
.
47
Ribeiro
CJN
,
Lima
A
,
de Araujo
RAS
, et al
.
Psychometric properties of the behavioral pain scale in traumatic brain injury
.
Pain Manag Nurs
.
2019
;
20
(
2
):
152
157
.
48
Chanques
G
,
Pohlman
A
,
Kress
JP
, et al
.
Psychometric comparison of three behavioural scales for the assessment of pain in critically ill patients unable to self-report
.
Crit Care
.
2014
;
18
(
5
):
R160
.
49
Liu
Y
,
Li
L
,
Herr
K
.
Evaluation of two observational pain assessment tools in Chinese critically ill patients
.
Pain Med
.
2015
;
16
(
8
):
1622
1628
.
50
Rijkenberg
S
,
Stilma
W
,
Bosman
RJ
,
van der Meer
NJ
,
van der Voort
PHJ
.
Pain measurement in mechanically ventilated patients after cardiac surgery: comparison of the Behavioral Pain Scale (BPS) and the Critical-Care Pain Observation Tool (CPOT)
.
J Cardiothorac Vasc Anesth
.
2017
;
31
(
4
):
1227
1234
.
51
Chanques
G
,
Jaber
S
,
Barbotte
E
, et al
.
Impact of systematic evaluation of pain and agitation in an intensive care unit
.
Crit Care Med
.
2006
;
34
(
6
):
1691
1699
.
52
Radtke
FM
,
Heymann
A
,
Franck
M
, et al
.
How to implement monitoring tools for sedation, pain and delirium in the intensive care unit: an experimental cohort study
.
Intensive Care Med
.
2012
;
38
(
12
):
1974
1981
.
53
Williams
TA
,
Martin
S
,
Leslie
G
, et al
.
Duration of mechanical ventilation in an adult intensive care unit after introduction of sedation and pain scales
.
Am J Crit Care
.
2008
;
17
(
4
):
349
356
.
54
de Jong
A
,
Molinari
N
,
de Lattre
S
, et al
.
Decreasing severe pain and serious adverse events while moving intensive care unit patients: a prospective interventional study (the NURSE-DO project)
.
Crit Care
.
2013
;
17
(
2
):
R74
.
55
Olsen
BF
,
Rustoen
T
,
Sandvik
L
,
Miaskowski
C
,
Jacobsen
M
,
Valeberg
BT
.
Implementation of a pain management algorithm in intensive care units and evaluation of nurses’ level of adherence with the algorithm
.
Heart Lung
.
2015
;
44
(
6
):
528
533
.
56
Olsen
BF
,
Rustoen
T
,
Sandvik
L
,
Jacobsen
M
,
Valeberg
BT
.
Results of implementing a pain management algorithm in intensive care unit patients: the impact on pain assessment, length of stay, and duration of ventilation
.
J Crit Care
.
2016
;
36
:
207
211
.
57
Gélinas
C
.
Development and validation of the Critical-Care Pain Observation Tool for critically ill adults presenting with different levels of consciousness (Tome 1 & Tome 2) [dissertation in French]
.
Québec, QC, Canada
:
Université Laval
;
2004
.
58
Gélinas
C
,
Fillion
L
,
Puntillo
KA
,
Viens
C
,
Fortier
M
.
Validation of the Critical-Care Pain Observation Tool in adult patients
.
Am J Crit Care
.
2006
;
15
(
4
):
420
427
.
59
Gélinas
C
,
Fillion
L
,
Puntillo
KA
.
Item selection and content validity of the Critical-Care Pain Observation Tool for non-verbal adults
.
J Adv Nurs
.
2009
;
65
(
1
):
203
216
.
60
Gélinas
C
,
Johnston
C
.
Pain assessment in the critically ill ventilated adult: validation of the Critical-Care Pain Observation Tool and physiologic indicators
.
Clin J Pain
.
2007
;
23
(
6
):
497
505
.
61
Gélinas
C
,
Rémy
J
,
Gagné
L
,
Desjardins
S
,
Chabot
MP
,
Vaillant
F
.
The CPOT: Assessing pain in unconscious adult patients [in French]
.
Perspective Infirmière
.
2015
;
12
(
2
):
48
53
.
62
Gélinas
C
.
Nurses’ evaluations of the feasibility and the clinical utility of the Critical-Care Pain Observation Tool
.
Pain Manag Nurs
.
2010
;
11
(
2
):
115
125
.
63
Frandsen
JB
,
O’Reilly Poulsen
KS
,
Laerkner
E
,
Stroem
T
.
Validation of the Danish version of the Critical Care Pain Observation Tool
.
Acta Anaesthesiol Scand
.
2016
;
60
(
9
):
1314
1322
.
64
Stilma
W
,
Rijkenberg
S
,
Feijen
HM
,
Maaskant
JM
,
Endeman
H
.
Validation of the Dutch version of the Critical-Care Pain Observation Tool
.
Nurs Crit Care
.
2019
;
24
(
3
):
132
140
.
65
Emsden
C
,
Schafer
UB
,
Denhaerynck
K
,
Grossmann
F
,
Frei
IA
,
Kirsch
M
.
Validating a pain assessment tool in heterogeneous ICU patients: is it possible?
[published online August 9, 2019]
Nurs Crit Care
. doi:
66
Mamoura
K
.
Validation of the Greek Version of the Critical Care Observation Tool [master’s thesis]
.
Athens, Greece
:
National and Kapodistrian University of Athens
;
2017
.
67
Yamada
S
,
Ikematsu
Y
.
Reliability, validity and responsiveness about the Japanese version of the Critical-Care Pain Observation Tool (CPOT-J)
.
J Jpn Soc Intensive Care Med
.
2016
;
23
(
2
):
133
140
.
68
Kwak
EM
,
Oh
H
.
Validation of a Korean translated version of the Critical Care Pain Observation Tool (CPOT) for ICU patients [in Korean]
.
J Korean Acad Nurs
.
2012
;
42
(
1
):
76
84
.
69
Chookalayia
H
,
Heidarzadeh
M
,
Hassanpour-Darghah
M
,
Aghamohammadi-Kalkhoran
M
,
Karimollahi
M
.
The Critical Care Pain Observation Tool is reliable in non-agitated but not in agitated intubated patients
.
Intensive Crit Care Nurs
.
2018
;
44
:
123
128
.
70
Kotfis
K
,
Zegan-Baranska
M
,
Strzelbicka
M
, et al
.
Validation of the Polish version of the Critical Care Pain Observation Tool (CPOT) to assess pain intensity in adult, intubated intensive care unit patients: the POL-CPOT study
.
Arch Med Sci
.
2018
;
14
(
4
):
880
889
.
71
Aktas
YY
,
Karabulut
N
.
A Turkish version of the Critical-Care Pain Observation Tool: reliability and validity assessment
.
J Perianesth Nurs
.
2017
;
32
(
4
):
341
351
.
72
Vazquez
M
,
Pardavila
MI
,
Lucia
M
,
Aguado
Y
,
Margall
MA
,
Asiain
MC
.
Pain assessment in turning procedures for patients with invasive mechanical ventilation
.
Nurs Crit Care
.
2011
;
16
(
4
):
178
185
.
73
Stefani
F
,
Nardon
G
,
Bonato
R
,
Modenese
A
,
Novello
C
,
Ferrari
R
.
The validation of C-POT (Critical-Care Pain Observation Tool) scale: a tool for assessing pain in intensive care patients [in Italian]
.
Assist Inferm Ric
.
2011
;
30
(
3
):
135
143
.
74
Storsveen
A-M
,
Hall-Lord
M-L
.
The CPOT—a tool for pain assessment for intensive care patients
.
2016
. .
75
Nürnberg Damstrom
D
,
Saboonchi
F
,
Sackey
PV
,
Bjorling
G
.
A preliminary validation of the Swedish version of the Critical-Care Pain Observation Tool in adults
.
Acta Anaesthesiol Scand
.
2011
;
55
(
4
):
379
386
.
76
Li
Q
,
Wan
X
,
Gu
C
, et al
.
Pain assessment using the Critical-Care Pain Observation Tool in Chinese critically ill ventilated adults
.
J Pain Symptom Manage
.
2014
;
48
(
5
):
975
982
.
77
Buttes
P
,
Keal
G
,
Cronin
SN
,
Stocks
L
,
Stout
C
.
Validation of the Critical-Care Pain Observation Tool in adult critically ill patients
.
Dimens Crit Care Nurs
.
2014
;
33
(
2
):
78
81
.
78
Chen
J
,
Hu
F
,
Yang
J
, et al
.
Validation of a Chinese version Critical-Care Pain Observation Tool in nonintubated and intubated critically ill patients: two cross-sectional studies
.
J Clin Nurs
.
2019
;
28
(
15–16
):
2824
2832
.
79
Dale
CM
,
Prendergast
V
,
Gélinas
C
,
Rose
L
.
Validation of The Critical-Care Pain Observation Tool (CPOT) for the detection of oral-pharyngeal pain in critically ill adults
.
J Crit Care
.
2018
;
48
:
334
338
.
80
Gélinas
C
,
Arbour
C
.
Behavioral and physiologic indicators during a nociceptive procedure in conscious and unconscious mechanically ventilated adults: similar or different?
J Crit Care
.
2009
;
24
(
4
):
628.e627
617
.
81
Gélinas
C
,
Tousignant-Laflamme
Y
,
Tanguay
A
,
Bourgault
P
.
Exploring the validity of the bispectral index, the Critical-Care Pain Observation Tool and vital signs for the detection of pain in sedated and mechanically ventilated critically ill adults: a pilot study
.
Intensive Crit Care Nurs
.
2011
;
27
(
1
):
46
52
.
82
Kanji
S
,
MacPhee
H
,
Singh
A
, et al
.
Validation of the Critical Care Pain Observation Tool in critically ill patients with delirium: a prospective cohort study
.
Crit Care Med
.
2016
;
44
(
5
):
943
947
.
83
Khanna
P
,
Pandey
RK
,
Chandralekha
C
,
Sharma
A
,
Pangasa
N
.
Comparison between Critical-Care Pain Observation Tool and physiologic indicators for pain assessment in the critically ill, mechanically ventilated adult patients
.
Saudi J Anaesth
.
2018
;
12
(
3
):
384
388
.
84
Paulson-Conger
M
,
Leske
J
,
Maidl
C
,
Hanson
A
,
Dziadulewicz
L
.
Comparison of two pain assessment tools in nonverbal critical care patients
.
Pain Manag Nurs
.
2011
;
12
(
4
):
218
224
.
85
Ross
M
,
Boitor
M
,
Gélinas
C
.
Validation of the Critical-Care Pain Observation Tool with seriously ill patients
.
J Hosp Palliat Nurs
.
2016
;
18
(
5
):
413
420
.
86
Boitor
M
,
Fiola
JL
,
Gélinas
C
.
Validation of the Critical-Care Pain Observation Tool and vital signs in relation to the sensory and affective components of pain during mediastinal tube removal in postoperative cardiac surgery intensive care unit adults
.
J Cardiovasc Nurs
.
2016
;
31
(
5
):
425
432
.
87
Faritous
Z
,
Barzanji
A
,
Azarfarin
R
, et al
.
Comparison of bispectral index monitoring with the Critical-Care Pain Observation Tool in the pain assessment of intubated adult patients after cardiac surgery
.
Anesth Pain Med
.
2016
;
6
(
4
):
e38334
.
88
Ghanbari
A
,
Bahadorizadeh
N
,
Farmanbar
R
,
Kazemnejad
E
.
Sensitivity and specificity of Critical Care Pain Observation Tool used for intubated patients following open-heart surgery
.
Intensive Care Med
.
2016
;
42
(
4
):
618
619
.
89
Keane
KM
.
Validity and reliability of the critical care pain observation tool: a replication study
.
Pain Manag Nurs
.
2013
;
14
(
4
):
e216
e225
.
90
Kiavar
M
,
Azarfarin
R
,
Totonchi
Z
,
Tavakoli
F
,
Alizadehasl
A
,
Teymouri
M
.
Comparison of two pain assessment tools, “Facial Expression” and “Critical Care Pain Observation Tool” in intubated patients after cardiac surgery
.
Anesth Pain Med
.
2016
;
6
(
1
):
e33434
.
91
Linde
SM
,
Badger
JM
,
Machan
JT
, et al
.
Reevaluation of the Critical-Care Pain Observation Tool in intubated adults after cardiac surgery
.
Am J Crit Care
.
2013
;
22
(
6
):
491
497
.
92
Echegaray-Benites
C
,
Kapoustina
O
,
Gélinas
C
.
Validation of the use of the Critical-Care Pain Observation Tool (CPOT) with brain surgery patients in the neurosurgical intensive care unit
.
Intensive Crit Care Nurs
.
2014
;
30
(
5
):
257
265
.
93
Joffe
AM
,
McNulty
B
,
Boitor
M
,
Marsh
R
,
Gélinas
C
.
Validation of the Critical-Care Pain Observation Tool in brain-injured critically ill adults
.
J Crit Care
.
2016
;
36
:
76
80
.
94
Lee
K
,
Oh
H
,
Suh
Y
,
Seo
W
.
Patterns and clinical correlates of pain among brain injury patients in critical care assessed with the critical care pain observation tool
.
Pain Manag Nurs
.
2013
;
14
(
4
):
259
267
.
95
Shan
K
,
Cao
W
,
Yuan
Y
, et al
.
Use of the Critical-Care Pain Observation Tool and the bispectral index for the detection of pain in brain-injured patients undergoing mechanical ventilation: a STROBE-compliant observational study
.
Medicine (Baltimore)
.
2018
;
97
(
22
):
e10985
.
96
Sulla
F
,
De Souza Ramos
N
,
Terzi
N
, et al
.
Validation of the Italian version of the Critical Pain Observation Tool in brain-injured critically ill adults
.
Acta Biomed
.
2017
;
88
(
5S
):
48
54
.
97
Topolovec-Vranic
J
,
Gélinas
C
,
Li
Y
, et al
.
Validation and evaluation of two observational pain assessment tools in a trauma and neurosurgical intensive care unit
.
Pain Res Manag
.
2013
;
18
(
6
):
e107
114
. doi:
98
Gélinas
C
.
When and how to evaluate interrater reliability of patient assessment tools
.
AACN Adv Crit Care
.
2011
;
22
(
4
):
412
417
.
99
Streiner
DL
,
Norman
GR
,
Cairney
J
.
Health Measurement Scales: A Practical Guide to Their Development and Use
. 5th ed.
New York, NY
:
Oxford University Press
;
2015
.
100
Gélinas
C
,
Harel
F
,
Fillion
L
,
Puntillo
KA
,
Johnston
CC
.
Sensitivity and specificity of the Critical-Care Pain Observation Tool for the detection of pain in intubated adults after cardiac surgery
.
J Pain Symptom Manage
.
2009
;
37
(
1
):
58
67
.
101
Bourbonnais
FF
,
Malone-Tucker
S
,
Dalton-Kischei
D
.
Intensive care nurses’ assessment of pain in patients who are mechanically ventilated: how a pilot study helped to influence practice
.
Can J Crit Care Nurs
.
2016
;
27
(
3
):
24
29
.
102
Gélinas
C
,
Ross
M
,
Boitor
M
,
Desjardins
S
,
Vaillant
F
,
Michaud
C
.
Nurses’ evaluations of the CPOT use at 12-month post-implementation in the intensive care unit
.
Nurs Crit Care
.
2014
;
19
(
6
):
272
280
.
103
Asadi-Noghabi
AA
,
Gholizadeh
M
,
Zolfaghari
M
,
Mehran
A
,
Sohrabi
M
.
Nurses use of Critical Care Pain Observational Tool in patients with low consciousness
.
Oman Med J
.
2015
;
30
(
4
):
276
282
.
104
Damico
V
,
Cazzaniga
F
,
Murano
L
,
Ciceri
R
,
Nattino
G
,
Dal Molin
A
.
Impact of a clinical therapeutic intervention on pain assessment, management, and nursing practices in an intensive care unit: a before-and-after study
.
Pain Manag Nurs
.
2018
;
19
(
3
):
256
266
.
105
Gélinas
C
,
Arbour
C
,
Michaud
C
,
Vaillant
F
,
Desjardins
S
.
Implementation of the Critical-Care Pain Observation Tool on pain assessment/management nursing practices in an intensive care unit with nonverbal critically ill adults: a before and after study
.
Int J Nurs Stud
.
2011
;
48
(
12
):
1495
1504
.
106
Phillips
ML
,
Kuruvilla
V
,
Bailey
M
.
Implementation of the Critical Care Pain Observation Tool increases the frequency of pain assessment for noncommunicative ICU patients
.
Aust Crit Care
.
2018
;
32
:
367
372
.
107
Mascarenhas
M
,
Beattie
M
,
Roxburgh
M
,
MacKintosh
J
,
Clarke
N
,
Srivastava
D
.
Using the Model for Improvement to implement the Critical-Care Pain Observation Tool in an adult intensive care unit
.
BMJ Open Qual
.
2018
;
7
(
4
):
e000304
.
108
Rose
L
,
Haslam
L
,
Dale
C
,
Knechtel
L
,
McGillion
M
.
Behavioral pain assessment tool for critically ill adults unable to self-report pain
.
Am J Crit Care
.
2013
;
22
(
3
):
246
255
.
109
Faust
AC
,
Rajan
P
,
Sheperd
LA
,
Alvarez
CA
,
McCorstin
P
,
Doebele
RL
.
Impact of an analgesia-based sedation protocol on mechanically ventilated patients in a medical intensive care unit
.
Anesthes Analg
.
2016
;
123
(
4
):
903
909
.
110
Arbour
C
,
Gélinas
C
,
Michaud
C
.
Impact of the implementation of the critical-care pain observation tool (CPOT) on pain management and clinical outcomes in mechanically ventilated trauma intensive care unit patients: a pilot study
.
J Trauma Nurs
.
2011
;
18
(
1
):
52
60
.
111
Latorre-Marco
I
,
Solís Munoz
M
,
Falero Ruiz
T
, et al
.
Validation of the Scale of Behavior Indicators of Pain (ESCID) in critically ill, non-communicative patients under mechanical ventilation: Results of the ESCID scale [in Spanish]
.
Enferm Intensiva
.
2011
;
22
(
1
):
3
12
.
112
Erdek
MA
,
Pronovost
PJ
.
Improving assessment and treatment of pain in the critically ill
.
Int J Qual Health Care
.
2004
;
16
(
1
):
59
64
.
113
Pardo
C
,
Munoz
T
,
Chamorro
C
Analgesia and Sedation Work Group of SEMICYUC
.
Monitoring pain: recommendations of the Analgesia and Sedation Work Group of SEMICYUC [in Spanish]
.
Med Intensiva
.
2006
;
30
(
8
):
379
385
.
114
Latorre-Marco
I
,
Acevedo-Nuevo
M
,
Solis-Munoz
M
, et al
.
Psychometric validation of the behavioral indicators of pain scale for the assessment of pain in mechanically ventilated and unable to self-report critical care patients
.
Med Intensiva
.
2016
;
40
(
8
):
463
473
.
115
Lopez-Lopez
C
,
Perez-Perez
T
,
Beneit-Montesinos
JV
, et al
.
Pain assessment in mechanically ventilated, non-communicative severe trauma patients
.
J Trauma Nurs
.
2018
;
25
(
1
):
49
59
.
116
Klein
DG
,
Dumpe
M
,
Katz
E
,
Bena
J
.
Pain assessment in the intensive care unit: development and psychometric testing of the nonverbal pain assessment tool
.
Heart Lung
.
2010
;
39
(
6
):
521
528
.
117
Odhner
M
,
Wegman
D
,
Freeland
N
,
Steinmetz
A
,
Ingersoll
GL
.
Assessing pain control in nonverbal critically ill adults
.
Dimens Crit Care Nurs
.
2003
;
22
(
6
):
260
267
.
118
Kabes
AM
,
Graves
JK
,
Norris
J
.
Further validation of the nonverbal pain scale in intensive care patients
.
Crit Care Nurse
.
2009
;
29
(
1
):
59
66
.
119
Kaya
P
,
Erden
S
.
Cross-cultural adaptation, validity and reliability of the Turkish version of Revised Nonverbal Pain Scale
.
Agri
.
2019
;
31
(
1
):
15
22
.
120
Chookalayi
H
,
Heidarzadeh
M
,
Hasanpour
M
,
Jabrailzadeh
S
,
Sadeghpour
F
.
A study on the psychometric properties of revised-nonverbal pain scale and original-nonverbal pain scale in Iranian nonverbal-ventilated patients
.
Indian J Crit Care Med
.
2017
;
21
(
7
):
429
435
.
121
Topolovec-Vranic
J
,
Canzian
S
,
Innis
J
,
Pollmann-Mudryj
MA
,
McFarlan
AW
,
Baker
AJ
.
Patient satisfaction and documentation of pain assessments and management after implementing the adult nonverbal pain scale
.
Am J Crit Care
.
2010
;
19
(
4
):
345
355
.
122
Sacco
TL
,
LaRiccia
B
.
Interprofessional implementation of a pain/sedation guideline on a trauma intensive care unit
.
J Trauma Nurs
.
2016
;
23
(
3
):
156
164
.
123
Puntillo
KA
,
Stannard
D
,
Miaskowski
C
,
Kehrle
K
,
Gleeson
S
.
Use of a pain assessment and intervention notation (P.A.I.N.) tool in critical care nursing practice: nurses’ evaluations
.
Heart Lung
.
2002
;
31
(
4
):
303
314
.
124
Merkel
SI
,
Voepel-Lewis
T
,
Shayevitz
JR
,
Malviya
S
.
The FLACC: a behavioral scale for scoring postoperative pain in young children
.
Pediatr Nurs
.
1997
;
23
(
3
):
293
297
.
125
Voepel-Lewis
T
,
Zanotti
J
,
Dammeyer
JA
,
Merkel
S
.
Reliability and validity of the face, legs, activity, cry, consolability behavioral tool in assessing acute pain in critically ill patients
.
Am J Crit Care
.
2010
;
19
(
1
):
55
61
.
126
McGuire
DB
,
Reifsnyder
J
,
Soeken
K
,
Kaiser
KS
,
Yeager
KA
.
Assessing pain in nonresponsive hospice patients: development and preliminary testing of the multidimensional objective pain assessment tool (MOPAT)
.
J Palliat Med
.
2011
;
14
(
3
):
287
292
.
127
Wiegand
DL
,
Wilson
T
,
Pannullo
D
, et al
.
Measuring acute pain over time in the critically ill using the Multidimensional Objective Pain Assessment Tool (MOPAT)
.
Pain Manag Nurs
.
2018
;
19
(
3
):
277
287
.
128
Chatelle
C
,
Majerus
S
,
Whyte
J
,
Laureys
S
,
Schnakers
C
.
A sensitive scale to assess nociceptive pain in patients with disorders of consciousness
.
J Neurol Neurosurg Psychiatry
.
2012
;
83
(
12
):
1233
1237
.
129
Schnakers
C
,
Chatelle
C
,
Vanhaudenhuyse
A
, et al
.
The Nociception Coma Scale: a new tool to assess nociception in disorders of consciousness
.
Pain
.
2010
;
148
(
2
):
215
219
.
130
Chatelle
C
,
De Val
MD
,
Catano
A
, et al
.
Is the nociception coma scale-revised a useful clinical tool for managing pain in patients with disorders of consciousness?
Clin J Pain
.
2016
;
32
(
4
):
321
326
.
131
Taber
KS
.
The use of Cronbach’s alpha when developing and reporting research instruments in science education
.
Res Sci Educ
.
2018
;
48
(
6
):
1273
1296
.
132
Fischer
JE
,
Bachmann
LM
,
Jaeschke
R
.
A readers’ guide to the interpretation of diagnostic test properties: clinical example of sepsis
.
Intensive Care Med
.
2003
;
29
(
7
):
1043
1051
.
133
Gélinas
C
,
Boitor
M
,
Puntillo
KA
, et al
.
Behaviors indicative of pain in brain-injured adult patients with different levels of consciousness in the intensive care unit
.
J Pain Symptom Manage
.
2019
;
57
(
4
):
761
773
.
134
Roulin
MJ
,
Ramelet
AS
.
Behavioral changes in brain-injured critical care adults with different levels of consciousness during nociceptive stimulation: an observational study
.
Intensive Care Med
.
2014
;
40
(
8
):
1115
1123
.
135
Gelinas
C
,
Richebe
P
.
Exploring the use of an innovative technology for pain assessment during mediastinal tube removal in cardiac surgery patients in the intensive care unit: the Nociception Level (NOL)
.
Intensive Care Med Experimental
.
2018
;
6
(
suppl 2
):
1319
.
136
Shahiri
S
,
Richard-Lalonde
M
,
Richebe
P
,
Gelinas
C
.
Exploration of the Nociception Level Index for pain assessment during endotracheal suctioning in mechanically ventilated patients in the intensive care unit
.
Can J Pain
.
2019
;
3
(
1
):
A131
.

Footnotes

The authors declare no conflicts of interest.