Machine Learning Algorithm for Accurate, Automated, Real-Time Prediction of Surgical Site Infections using EHR Data

Abstract

Introduction: Identification of those patients who will develop surgical site infections (SSI) is desirable as SSI is a major cause of morbidity and readmission affecting 2-11% of general surgery patients. Current predictive models for SSI have limited use due to insufficient accuracy or because they are not practical for deployment. Our goal is to perform automated, real-time, accurate risk prediction for SSI through analysis of discretely encoded data in the electronic health record (EHR). Methods: Wederived an automated Naive Bayes prediction model using data from general surgery encounters in a single surgical unit from January 2011 to June 2012, and whose SSI outcomes were submitted to the National Surgical Quality Improvement Program (NSQIP). The primary outcome measured was postoperative SSI (superficial, deep, or organ space) within 30 days of operation. All patients in this data set were included in the study. All predictor variables used were available or calculated in real-time during the patient's hospitalization and included age, sex, ethnicity, zip code, surgical Apgar score, preoperative hemoglobin, last recorded hemoglobin during encounter, estimated blood loss, transfusion volume, ASA score, surgeon, and body mass index. Repeated random subsampling was used for cross-validation. Model discrimination, calibration, accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were examined. The model was then validated against patients who underwent general surgery procedures in the same unit from July 2012 to December 2012. All models were built using free, open-source R language (version 2.15). Results: The derivation set had 1302 patient encounters, and the validation set had 343 patient encounters, for a total of 1645 encounters. Of these, 113 (8.7%) in the derivation set and 30 (8.7%) in the validation set developed postoperative SSI within 30 days of operation. The model discrimination, accuracy, sensitivity, specificity, PPV, NPV, and calibration error for the derivation set were 0.89, 0.88, 0.72, 0.89, 0.41,0.97, and 0.08 respectively; and for the validation set, were 0.90, 0.87, 0.73, 0.88, 0.37, 0.97, and 0.01. Conclusions: We have demonstrated that automated and highly accurate predictionofSSI canbe accomplished by real-timeEHRanalysisusing amachine-learning algorithm. This is the highest performingmodel in the literature that we are aware of for predicting SSI in general surgery patients,with nearly 90%accuracy and 72%sensitivity. Such automated analysis can provide practical and meaningful SSI risk assessment during a patient's hospitalization. Additionally, because it is a machine-learning algorithm, it may easily be recalibrated using new data as the patient care environment changes. When combined with appropriate operational rules for patient care, it may allow for more selective use of high-resource or expensive interventions for decreasing surgical site infections.