Developing and Validating an Emergency Triage Model Using Machine Learning Algorithms with Medical Big Data

Abstract

Objective: To establish an emergency triage model through the statistical analysis of big data during a particular time period from a hospital information system to improve the accuracy of triage in emergency department (ED). Methods: A total of 276,164 patients who visited the Emergency Medicine Department of Beijing Chao-Yang Hospital from 2017 to 2020 were included in this study, including 123,392 men and 152,772 women aged from 14 to 112 years. The baseline characteristics (age and gender) and medical records (patient’s condition, body temperature, heart rate, breathing, blood pressure, consciousness, and oxygen saturation) of the patients was collected. The data samples were randomly allocated, with 80% as the training set and 20% as the testing set. The patients were divided into levels I, II, III, and IV in accordance with a four-level triage standard. We selected the effective Extreme Gradient Boosting (XGBoost) algorithm as our emergency classification prediction model. The XGBoost model was applied to simulate the thinking process of triage nurses, and the De Long’s test was used to compare the receiver operating characteristic (ROC) curve of different models. The P value was obtained by calculating the variance and covariance of area under the curve (AUC) values of different ROC curves. Results: Level I had 4960 (1.8%) patients, level II had 25,646 (9.29%), level III had 130,664 (47.31%), and level IV had 114,894 (41.6%). The XGBoost model was built following a logic exercise based on the traditional manual pre-inspection and triage results. After verification, the prediction accuracy was 82.57%. The AUC of each disease severity level (levels I, II, III, and IV) was 0.9629, 0.9554, 0.9120, and 0.9296, respectively. Conclusion: The emergency triage prediction model, which achieved a relatively strong accuracy rate, can reduce the work intensity of medical workers and improve their working efficiency.