Deep learning based multimedia data mining for autism spectrum disorder (ASD) diagnosis

Abstract

Autism Spectrum Disorder (ASD) is a neuro-developmental disorder characterized by deficits in social communication and restricted and repetitive patterns of behavior. Autism is estimated to affect 1 in 59 children in the United States and costs roughly 35B to the society. Early diagnosis of ASD is vital for promoting early intervention and positive developmental outcomes. Traditional diagnostic procedures for ASD include structured behavioral observation by a trained clinician. Diagnosticians typically rely on the Autism Diagnostic Observation Schedule (ADOS-2) to quantify ASD symptoms. In this paper, we take a parallel approach and investigate language modalities and discover associations between objective measurements of social communication and ASD symptoms. We analyze 33 children with autism and extract their linguistic patterns from their conversations with diagnosticians in a clinical setting. Our methods use Long-Short Term Memory (LSTM) networks to learn Speech Activity Detection (SAD) and speaker diarization patterns to generate the vocal turn-taking metrics. We then use our novel proposed pipeline to predict the ADOS-2 Calibrated Severity Scores (CSS) of Social Affect (SA). The proposed framework achieve state-of-the-art predictive diagnostic estimates of ASD severity compared to industry's leading algorithms. Results compared with the language acquisition system Language ENvironment Analysis (LENA) and other algorithms indicate a significant improvement in the R2 measure.