Technology assessment in critical care: understanding statistical analyses used to assess agreement between methods of clinical measurement

Many new measurement methods that employ various technologies to measure physiological parameters have been introduced into the field of critical care. Clinical assessment of these new methods occurs through the conduct of method-comparison studies in which the level of agreement between a new measurement method and a clinical standard method is determined. Clinicians and researchers are often faced with the complicated task of analyzing and interpreting the results of method-comparison studies. Use of correlation and linear regression techniques has been prevalent in method-comparison studies but has proven inappropriate and inadequate in determining how well methods compare. The purposes of this article are to briefly review the terms of accuracy, agreement, and the precision in context with method-comparison studies, and discuss inappropriate and appropriate statistical analyses and their interpretation. Appropriate data analysis of method-comparison studies will aid in determining not only whether new monitoring methods can be interchanged or used in place of existing methods, but whether new methods warrant further research of their effect on patient outcomes.