2160Performance of a machine learning model vs. IMPROVE score for VTE prediction in acute medically ill patients: insights from the APEX trial

Abstract

Background: Acutely ill hospitalized medical patients have an increased risk for venous thromboembolism (VTE). Current risk assessment models (RAM) have demonstrated modest performance, at best, in predicting the occurrence of VTE in these patients. Purpose: Evaluate the discrimination and calibration performance of a machine learning model in estimating VTE risk compared to the recommended IMPROVE score. Methods: The APEX trial was a multicenter, double‐blind, placebo‐controlled trial that randomized 7,513 hospitalized acutely ill medical patients to extended duration betrixaban vs. standard of care enoxaparin. Patients were followed for up to 77 days. A super learner algorithm was built to predict VTE through 77 days after hospitalization by combining generalized additive models (GAMs), LASSO and RIDGE regressions, Random Forests, and Gradient Boosted Machines. The IMPROVE score was calculated for each patient to serve as a comparator. C‐statistics were used to evaluate discrimination and a boostrapped significance test was performed to calculate a p‐value for the difference. The Hosmer‐ Lemeshow goodness‐of‐fit test and calibration curves assessed the reliability of the RAMs. Results: The super learner algorithm significantly outperformed the IMPROVE score in predicting VTE at 77 days after hospitalization for an acute medical illness (c‐statistic: 0.68 vs. 0.59; p‐value <0.001). Additionally, the super learner model demonstrated excellent calibration with a p‐value for the Hosmer‐Lemeshow of 0.60. Conversely, the IMPROVE score demonstrated poor calibration with a Hosmer‐Lemeshow p‐value of less than 0.001. Conclusion: The super learner algorithm demonstrated significantly improved discrimination and excellent calibration compared to the IMPROVE score for predicting VTE up to 77 days after hospitalization for an acute medical illness. (Figure Presented) .