Explainable AI for Medical Event Prediction for Heart Failure Patients

Abstract

The past decade has witnessed significant progress in deploying AI in the medical field. However, most AI models are considered black-boxes, making predictions neither understandable nor interpretable by humans. This limitation is especially significant when they contradict clinicians’ expectations based on medical knowledge. This can lead to a lack of trust in the model. In this work, we propose a pipeline to explain AI models. We used a previously devised Neural Network model to present our approach. It predicts the daily risk for patients with heart failure and is a part of a Decision Support System. In our pipeline, we deployed DeepSHAP algorithm to receive global and local explanations. With a global explanation, we defined the most important features in the model and their influence on the prediction. With local explanation, we analyzed individual observations and explained why a specific prediction was made. To validate the clinical relevance of our results, we consulted them with medical experts and made a literature review. Moreover, we described how the proposed pipeline can be integrated into Decision Support Systems. With the above tools, medical personnel can analyze the root of decisions and have insights into how medical parameters should be changed to improve the patient’s health state.