Development of an iterative validation process for a 30-day hospital readmission prediction index

Abstract

Purpose. A study was conducted to determine if an iterative validation process could maintain or improve the discriminative and predictive capabilities of a 30-day hospital readmission prediction index over 2.5 years. Methods. Patient admissions were retrospectively identified using the electronic medical record. The receiver operating characteristic curve was used to assess model discrimination. Prediction index specificity, sensitivity, and positive and negative predictive values were also assessed. A rolling iterative validation process was developed in which patient admissions were divided into 3-month cohorts. Each cohort was analyzed individually and then included into the cumulative patient cohort and analyzed again. Results. From 121,277 patient visits, an iterative validation approach maintained the discrimination (0.71 to 0.72), predictive validity, and overall accuracy (80.9% to 81.7%) of the 30-day readmission prediction index over 2.5 years. Index sensitivity and negative predictive value increased from baseline while specificity and positive predictive value remained largely unchanged. None of the assessed index parameters diminished or became less useful over the course of the study. Conclusion. An internal iterative validation process based on frequentist statistics maintained the discriminative ability and accuracy of a readmission index over 2.5 years despite numerous changes in the variables associated with readmission in the patient population.