Explainable AI predicting Alzheimer’s disease with latent multimodal deep neural networks

Abstract

Purpose: Alzheimer’s disease (AD) is a neurodegenerative disorder characterized by progressive cognitive decline. We proposed a novel latent multimodal deep learning framework to predict AD cognitive status using clinical, neuroimaging, and genetic data. Methods: Three hundred and twenty-two patients aged between 55 and 92 from the ADNI database were included in the study. Confirmatory Factor Analysis (CFA) was applied to derive the latent scores of AD cognitive impairments as the outcome. A multimodal deep neural network with three modalities, including clinical data, imaging data, and genetic data, was constructed. Attention layers and cross attention layers were added to improve prediction; modality importance scores were calculated for interpretation. Mean Absolute Error (MAE) and Mean Squared Error (MSE) were used to evaluate the model performance. Results: The CFA demonstrated good fit to the data. The multimodal neural network of clinical and imaging modalities with attention layers was the best predictive model, with an MAE of 0.330 and an MSE of 0.206. Clinical data contributed the most (35%) to the prediction of AD cognitive status. Conclusion: Our results demonstrated the attention multimodal model’s superior performance in predicting the cognitive impairment of AD, introducing attention layers into the model enhanced the prediction performance.