SpeechCARE: Harnessing Multimodal Innovation to Transform Cognitive Impairment Detection – Insights from the National Institute on Aging Alzheimer's Speech Challenge

Abstract

Alzheimer’s disease and related dementias (ADRD) affect one in five adults over 60, yet over half of individuals with cognitive decline remain undiagnosed. This study introduces SpeechCARE, a novel multimodal and multilingual speech processing pipeline leveraging pretrained speech and linguistic transformer models for the National Institute on Aging (NIA)’s PREPARE Challenge. Evaluated on a three-class dataset (Control, Mild Cognitive Impairment, Alzheimer’s) of 2,058 participants in three languages and multiple speech production tasks, SpeechCARE uses a novel modality fusion mechanism that dynamically weights modality inputs and integrates task-specific acoustic and linguistic cues. It achieved an AUC of 0.85 and an F1 score of 70.6, placing it among the finalists in the ongoing NIA PREPARE Challenge. By combining multilingual transformer models with its innovative fusion mechanism, SpeechCARE demonstrates robust adaptability across diverse languages, varied speech tasks, and complex real-world clinical settings.