A Simplified Score for Transfer of Patients Requiring Mechanical Ventilation to a Long-Term Care Hospital

We used hospital discharge and billing data from the 2005 Nationwide Inpatient Sample from the Healthcare Cost and Utilization Project.⁹  The 2005 Nationwide Inpatient Sample includes data for approximately 8 million hospital discharges at 1054 hospitals provided to generate nationally representative estimates. Overall, the sampling frame for the 2005 Nationwide Inpatient Sample comprised 75.0% of all US hospitals and encompassed 86.3% of the US population. Detailed information regarding the sampling frame and weighting scheme is provided elsewhere.⁹  The Health Sciences Institutional Review Board at the University of Wisconsin-Madison approved this study.

The study population consisted of all inpatient discharges during the study period, from January 2005 to December 2005, that met the following criteria: (1) primary Medicare coverage; (2) age 65 years or older at the time of admission; (3) received mechanical ventilation within 7 days of admission; (4) had an LOS of 7 days or longer. We excluded patients younger than 65 years because they are typically enrolled in Medicare because of disability or end-stage renal diseases, which makes them different from the major elderly population of Medicare beneficiaries.

We used patient information available on day 7 of admission to predict whether a patient receiving mechanical ventilation would or would not stay an additional 25 days (overall LOS ≥32 days; Figure 1). The model was not designed to predict whether patients would be in sufficiently stable condition for transfer on day 7 or thereafter.

Characteristics of patients and hospitals were considered as potential predictors. Patients’ characteristics, including age, sex, race, and admission type, were obtained from discharge records. Age was categorized in 5-year increments (65–69, 70–74, 75–79, 80–84, and ≥85 years). Hospitals’ characteristics, including teaching status (teaching vs nonteaching), location (urban vs rural), region (Northeast, West, South, Midwest), and bed size (small, medium, large)¹⁰  also were considered. Of all patients in the study, 25% did not report race and 11% did not report admission type. Missing race and admission type data were categorized as “unknown.” Patients were assessed for the presence of 29 comorbid conditions existing before hospital admission¹¹  and for the presence of 231 categories of procedure coded within 7 days of hospitalization.¹² 

We used split-sample validation, with 50% of the sample randomly assigned to the derivation data set and 50% to the validation data set. Differences in proportion of all dichotomous variables between the derivation data set and the validation data set were assessed by using χ² tests. All potential predictors were initially categorized into 3 groups: (1) patient and hospital characteristics, (2) comorbid conditions, and (3) procedures. Within each group, potential predictors were fitted into multivariable survey-logistic regression models¹³ ; statistically significant predictor variables from each group were retained. For the comparison between groups, the statistical significance was defined as P less than .05. For the regression analyses, a significance criterion of P less than .01 was chosen because of the large sample size and a desire to avoid overfitting. Interaction terms (“Renal failure × Hemodialysis” and “Tracheostomy × Gastrostomy”) were entered and tested in subsequent analyses. After defining our predictive model from the derivation data set, parameter estimates obtained from the derivation data set were applied to the validation data set. Results of the multivariable survey-logistic regression analysis were reported as odds ratios (OR) and 99% confidence intervals (CI).

We used 2 methods to assess the accuracy of our predictive model. First, we examined the ability of the model to correctly distinguish patients with an LOS of 32 days or longer from those with an LOS less than 32 days, using the area under the receiver operator characteristic curve, or C statistic. A C statistic of 0.5 (50%) indicates that the scale or tool has no ability to discriminate beyond chance, whereas a C statistic of 1.0 (100%) represents perfect discrimination.¹⁴  Second, we examined the model calibration by graphically displaying calibration curves from both data sets. The mean predicted probability of having an LOS of 32 days or longer, from 0 to 1, was categorized into 10 strata by 10% increments. Calibration curves were generated by plotting the mean predicted probability vs the mean observed probability of LOS of 32 days or longer across the probability stratum.

To generate a simplified score, important variables were selected from the final predictive model and were assigned points on the basis of their odds ratios. Each patient was assigned a score by adding up the points of risk factors present for that patient. The C statistic was assessed for the simplified score. By using different cutpoints, the scores were categorized into risk classes. All statistical analyses were done with SAS software version 9.1.3 (SAS Institute, Cary, North Carolina).

During the study period, 54 686 inpatient discharges met the inclusion criteria and were randomly selected into a derivation data set (n = 27 343) and a validation data set (n = 27 343; Table 1). Within our study sample, 10% had an LOS of 32 days or longer, 26% died during hospitalization, and 5% were transferred to LTCHs (4% before day 32, 1% after day 32). No statistically significant (P< .05) differences in the proportion of variables were found between the derivation data set and the validation data set, except for “admission type” (P= .04). The most prevalent comorbid conditions were chronic pulmonary disease (44%) and hypertension (44%). About 4% of the study population received a tracheostomy within 7 days of admission.

Twenty-two predictor variables met the significance criterion for inclusion in the final survey-logistic model (Table 2). Receiving a tracheostomy was the strongest predictive factor (OR = 4.74, 99% CI = 3.54–6.35, P< .001). Receiving a gastrostomy was also highly associated with the outcome (OR = 3.23, 99% CI = 2.50–4.16, P< .001). We found a significant negative interaction between receipt of tracheostomy and gastrostomy (OR = 0.28, 99% CI = 0.19–0.43, P < .001). Hence, receipt of both procedures is not likely to multiplicatively increase the risk of LOS of 32 days or greater.

Our model performed consistently in both the derivation and the validation data sets, with C statistics of 0.75 and 0.75, respectively. Figure 2 presents the calibration curves of the predictive model from the derivation and the validation data sets. In general, our model is well calibrated, except among patients with especially high risk (≥0.7) of being hospitalized longer than 32 days, where stochastic variation might be present because of the small number of individuals within the sample at such high risk (<0.2% of the data set).^15,16 

Table 3 presents the points assigned for each factor of the simplified score. The simplified score had a discriminatory power, or C statistic of 0.72. The simplified score had a range from −5 to 110 points, with an increment of 5 points. Sensitivity and specificity of our scoring tool depend on the chosen point threshold (“cutpoint”) at which a patient is deemed “positive” for likelihood of satisfying the LOS criterion for transfer to an LTCH. At a cutpoint of 30, the score predicted the risk of LOS of 32 days or longer with a sensitivity of 40.6% and a specificity of 86.0%. Use of 35 points as the cut-point reduced sensitivity to 33.9% and increased specificity to 90.1%. If we are willing to accept one false-positive result (ie, a patient predicted to satisfy the LTCH length of stay criterion at day 7 who is actually discharged before day 32) to get 2 true-positive results, then the cutpoint should be 35 points. Similarly, if we require 3 to 4 true-positive results to be willing to accept 1 false-positive result, then the cutpoint should be 60. The relative benefits of true-positive results versus the costs of false-positive results are likely to vary by institution. Thus, we categorized the score into 3 classes: less than 30 points as low risk, 30 to 60 points as intermediate risk, and 60 points or more as high risk.

Physicians and hospital administrators are increasingly under pressure to improve quality of care while monitoring economic performance. The purpose of our study was to develop a predictive model that would help identify patients receiving mechanical ventilation who were likely to fulfill the LOS criteria for transfer to LTCHs. By using a nationally representative hospital administrative data set, we were able to develop a well-calibrated risk-adjustment model with good discriminatory power. The predictive model developed in our study was simplified to devise a scoring tool that uses readily available variables to identify eligible patients for LTCH transfer. Because the model is based on information available on day 7 after admission, our approach informs real-time decision making. Clinicians can make referrals on the basis of the presence of those factors that have the strongest association with LOS of 32 days or longer. Our simplified index can be easily used at the bedside to monitor the patients on the basis of their risk level.

In this study, we found that various characteristics of patients and hospitals, comorbid conditions, and procedures were significant predictors associated with the outcome. Most importantly, these included tracheostomy, gastrostomy, the need for hemodialysis, wound debridement, physical therapy, and rehabilitation.

The indications for frequency and timing of tracheostomy are extremely variable in the United States and worldwide.¹⁷  Performance of a tracheostomy usually reflects the clinician’s judgment that prolonged ventilation or at least airway support will be necessary. MacIntyre et al¹⁸  recommend that clinicians consider referral to a facility focused on prolonged mechanical ventilation when a tracheostomy is first considered. This recommendation is consistent with our finding that tracheostomy was the strongest predictor for having an LOS of 32 days or longer. However, there is equipoise with respect to early vs late tracheostomy placement: a tracheostomy by day 7 of an inpatient stay is typically considered “early.” Given that early tracheostomies have, in some studies, been shown to reduce LOS and duration of mechanical ventilation,^19,20  the finding that tracheostomies are predictive of an LOS of 32 days or longer was not necessarily an expected one. But in a recent study,²¹  researchers reported a statistically significant increase in the number of ventilator-free and ICU free days in the early tracheostomy group, and the median hospital LOS in that group was 31 days (interquartile range, 17–49 days). Although the scope of our study is to provide a simple referral planning tool to use at a single point in time, nonetheless, whether timing of tracheostomy is a predictor of LOS is an important area for further research.

We found that patients aged 80 and over were significantly less likely to have an LOS of 32 days or longer. This difference could be due to a higher risk of in-hospital mortality, which would shorten LOS. We also found that race was a significant predictor, with blacks significantly more likely than whites to stay beyond 32 days. This finding may reflect a preference that African American patients are less likely to have do-not-resuscitate orders in place at the time of hospitalization²²  and are less likely to favor withdrawal of life-supporting measures in the ICU.²³ 

Hospital characteristics, such as location, region, bed size, and teaching status, are associated with LOS.^24,25  Researchers in a prior study²⁴  reported that teaching hospitals generally have a higher mean LOS than nonteaching hospitals have. However, we found no statistically significant association between hospital teaching status and the outcome. In our study, patients admitted to urban hospitals were more likely to have an LOS of 32 days or longer, after adjusting for case mix. We also found that patients admitted to hospitals in the Northeastern region of the country were more likely to have a longer LOS. The variance between regions might be explained by the availability of alternatives to acute care or differing case management strategies and clinical practice patterns.

Although bed size is positively related to average LOS,²⁴  we found that patients admitted to smaller hospitals are more likely to have an LOS of 32 days or longer than were patients admitted to larger hospitals. The discrepancy has 2 possible explanations. First, in the data from the 2005 Nationwide Inpatient Sample,¹⁰  bed size was categorized into 3 groups (ie, small, medium, large), with the criteria for each group set differently according to a hospital’s location, region, and teaching status. Second, it could be argued that large capacity hospitals have more resources and might be able to treat medically complex patients more effectively, resulting in shorter LOS.

Comorbid conditions traditionally have been regarded as important risk factors. Patients with comorbid conditions often have a longer LOS. However, we found that patients with chronic pulmonary disease or hypertension were less likely to have an LOS of 32 days or longer. Patients with chronic obstructive pulmonary disease may have a higher risk of mortality that could lead to a shorter LOS. On the other hand, risk-adjustment studies that used administrative data have shown a counterintuitive negative relationship between utilization and report of common chronic conditions such as hypertension.^11,26–28  Coding bias resulting from limited coding space may be at play, because in patients with numerous serious conditions, coders may not have room to report the patient’s more common conditions. Several strategies have been proposed for improving the accuracy of comorbidity measures that are based on claims data.²⁹  Although increasing the number of diagnoses coded might reduce bias, it is uncertain to what extent the problem of coding bias would be solved.³⁰ 

Although hemodialysis was significantly associated with the outcome of an LOS of 32 days or longer, we did not include it in the scoring scheme. It is uncommon for patients to be coded as receiving hemodialysis without having a code for renal failure. When we assigned a score for hemodialysis plus renal failure, the result was the same as renal failure alone. Because both scenarios receive the same score, our algorithm was simplified by assigning a value of 5 points to renal failure, regardless of whether or not dialysis was coded.

Several possible limitations must be considered while interpreting our results. Although administrative data bases contain demographic information, diagnoses, comorbid conditions, clinical services, and severity measures on large numbers of patients, these databases are limited by the lack of clinically important physiological information and the inability to differentiate between conditions present on admission and complications that occurred during hospitalization.^31,32  Also, we generally assume that administrative data provide reasonably valid information on diagnoses and clinical services. However, various factors, such as misdiagnoses, incomplete documentation of clinical information, or miscoding of diagnoses and procedures all unintentionally contribute errors.^11,26,33  Thus, the validity of risk-adjustment models based solely on administrative data has been challenged.^34–36 

A further limitation was the presence of missing data in our study. It is well known that racial identity is often not consistently reported. Although we found a higher odds of an LOS of 32 days or longer in African American patients (OR = 1.32, P= .002), this finding should be interpreted cautiously, given that 25% of discharges in our data set failed to report race. Our conclusions regarding racial differences in LOS are only suggestive and warrant further research. Similarly, although we found that “elective” admissions were more likely to have a longer LOS than were “emergency” admissions (OR = 1.90, P< .001), 11% of patients discharged did not report admission type in our data set.

Our study population included patients who died during hospitalization after day 7 (26%) and patients who were transferred to LTCHs before day 32 (4%). If characteristics associated with a longer LOS are also associated with high risk of mortality or actual transfer to an LTCH, their association with an LOS of 32 days or longer may be underestimated by including these patients. However, our model is designed to generate predictions by using only information available at day 7; since foreknowledge of death or LTCH transfer is not possible, exclusion of such patients would inappropriately change the study population. We explored the potential effect of right-hand censoring of actual LTCH transfers before 32 days by recoding the dependent variable to 1 for these individuals and repeating the regression analysis. The C statistic and estimates of odds ratios for predictor variables were largely unchanged, with the exception that the odds ratio for tracheotomy increased from 4.74 (P< .001) to 6.32 (P< .001).

Future work should examine the change in predictive performance associated with using information available at different time points (eg, day of admission, day 3, day 14, etc). Because the referral process can take up to 2 weeks for completion, optimization of the time at which the predictive model is used also becomes important in maximizing cost savings.

Long-term ventilator management and liberation from mechanical ventilation is a complex process that requires a multidisciplinary approach. Although the medical appropriateness for transfer is central to the ultimate discharge decision (and was not assessed in this analysis), our model and simplified index provide useful information for acute care physicians, nurses, and case managers to facilitate patients’ discharge planning. Through better planning of ICU bed occupancy, hospitals can improve patient flow, maximize throughput in the ICU, allocate resources more effectively, and eventually improve the efficiency of inpatient care.

The authors thank Dr Maureen Smith and Dr Mari Palta for their comments and suggestions throughout the duration of this study.