Neural network models for predicting readmission among patients undergoing peripheral vascular intervention using electronic health record data and clinical registry data

Abstract

Objectives To determine whether neural network models based on electronic health record (EHR) data can match and augment the performance of models based on clinical registry data in predicting readmission after peripheral vascular intervention (PVI). Design Observational cohort study. Setting Vascular Quality Initiative registry and INSIGHT Clinical Research Network EHR data from multiple academic institutions in New York City. Participants Patients undergoing PVI during January 1, 2013 to September 30, 2021. Main outcome measures Our outcome variable was 90-day readmission. We developed logistic regression (LR), multilevel perceptron (MLP), and recurrent neural network (RNN) models using registry alone, EHR data alone, and combined registry-EHR data. EHR data were evaluated using derived variables to match registry variables (EHR-derived data) and clinically meaningful code aggregation (EHR-direct data). Models were evaluated using area under the curve (AUC) for discrimination, Spiegelhalter z score for calibration, and Brier score for overall performance. Results The analytical cohort included 2348 patients undergoing PVI (mean age: 69.9±11.5 years). 832 (35%) patients were readmitted within 90 days. LR to predict 90-day readmission based on registry data alone had an AUC of 0.710, Spiegelhalter z score of 1.021, and Brier score of 0.211. MLP based on registry data alone had similar performance. MLP and RNN based on EHR-direct data (MLP: AUC=0.742, Spiegelhalter z=0.933, Brier=0.204; RNN: AUC=0.737, Spiegelhalter z=1.026, Brier=0.206) and registry+EHR-direct data (MLP: AUC=0.756, Spiegelhalter z=0.794, Brier=0.199; RNN: AUC=0.751, Spiegelhalter z=1.057, Brier=0.200) had improved performances. LR based on EHR-direct data and combined registry+EHRdirect data had worse performances. Conclusions EHR data, when used with neural network models, can be useful to establish readmission predictive models or augment clinical registry data. EHR-based models can be potentially embedded in the clinical workflow, but model performance may be constrained by the absence of certain information in clinical encounters, such as social determinants of health.