Validation of Machine Learning Models to Predict Adverse Outcomes in Patients with COVID-19: A Prospective Pilot Study

Abstract

Purpose: We previously developed learning models for predicting the need for intensive care and oxygen among patients with coronavirus disease (COVID-19). Here, we aimed to prospectively validate the accuracy of these models. Materials and Methods: Probabilities of the need for intensive care [intensive care unit (ICU) score] and oxygen (oxygen score) were calculated from information provided by hospitalized COVID-19 patients (n=44) via a web-based application. The performance of baseline scores to predict 30-day outcomes was assessed. Results: Among 44 patients, 5 and 15 patients needed intensive care and oxygen, respectively. The area under the curve of ICU score and oxygen score to predict 30-day outcomes were 0.774 [95% confidence interval (CI): 0.614–0.934] and 0.728 (95% CI: 0.559–0.898), respectively. The ICU scores of patients needing intensive care increased daily by 0.71 points (95% CI: 0.20–1.22) after hospitalization and by 0.85 points (95% CI: 0.36–1.35) after symptom onset, which were significantly different from those in individuals not needing intensive care (p=0.002 and <0.001, respectively). Trends in daily oxygen scores overall were not markedly different; however, when the scores were evaluated within <7 days after symptom onset, the patients needing oxygen showed a higher daily increase in oxygen scores [1.81 (95% CI: 0.48–3.14) vs.-0.28 (95% CI: 1.00–0.43), p=0.007]. Conclusion: Our machine learning models showed good performance for predicting the outcomes of COVID-19 patients and could thus be useful for patient triage and monitoring.