Hybrid artificial intelligence outcome prediction using features extraction from stress perfusion cardiac magnetic resonance images and electronic health records

Abstract

Background: Prediction of clinical outcomes in coronary artery disease (CAD) has been conventionally achieved using clinical risk factors. The relationship between imaging features and outcome is still not well understood. This study aims to use artificial intelligence to link image features with mortality outcome. Methods: A retrospective study was performed on patients who had stress perfusion cardiac magnetic resonance (SP-CMR) between 2011 and 2021. The endpoint was all-cause mortality. Convolutional neural network (CNN) was used to extract features from stress perfusion images, and multilayer perceptron (MLP) to extract features from electronic health records (EHRs), both networks were concatenated in a hybrid neural network (HNN) to predict study endpoint. Image CNN was trained to predict study endpoint directly from images. HNN and image CNN were compared with a linear clinical model using area under the curve (AUC), F1 scores, and McNemar’s test. Results: Total of 1,286 cases were identified, with 201 death events (16%). The clinical model had good performance (AUC =80%, F1 score =37%). Best Image CNN model showed AUC =72% and F1 score =38%. HNN outperformed the other two models (AUC =82%, F1 score =43%). McNemar’s test showed statistical difference between image CNN and both clinical model (P<0.01) and HNN (P<0.01). There was no significant difference between HNN and clinical model (P=0.15). Conclusions: Death in patients with suspected or known CAD can be predicted directly from stress perfusion images without clinical knowledge. Prediction can be improved by HNN that combines clinical and SP-CMR images.