Deep reinforcement learning extracts the optimal sepsis treatment policy from treatment records

Abstract

Background: Sepsis is one of the most life-threatening medical conditions. Therefore, many clinical trials have been conducted to identify optimal treatment strategies for sepsis. However, finding reliable strategies remains challenging due to limited-scale clinical tests. Here we tried to extract the optimal sepsis treatment policy from accumulated treatment records. Methods: In this study, with our modified deep reinforcement learning algorithm, we stably generated a patient treatment artificial intelligence model. As training data, 16,744 distinct admissions in tertiary hospitals were used and tested with separate datasets. Model performance was tested by t test and visualization of estimated survival rates. We also analyze model behavior using the confusion matrix, important feature extraction by a random forest decision tree, and treatment behavior comparison to understand how our treatment model achieves high performance. Results: Here we show that our treatment model’s policy achieves a significantly higher estimated survival rate (up to 10.03%). We also show that our models’ vasopressor treatment was quite different from that of physicians. Here, we identify that blood urea nitrogen, age, sequential organ failure assessment score, and shock index are the most different factors in dealing with sepsis patients between our model and physicians. Conclusions: Our results demonstrate that the patient treatment model can extract potential optimal sepsis treatment policy. We also extract core information about sepsis treatment by analyzing its policy. These results may not apply directly in clinical settings because they were only tested on a database. However, they are expected to serve as important guidelines for further research.