M-MS: A Multi-Modal Synchrony Dataset to Explore Dyadic Interaction in ASD

Abstract

Empathy, reciprocity and turn-taking are critical therapeutic targets in conditions of social impairment such as Autism Spectrum Disorder (ASD). These aspects are related to each other, converging into the construct of synchrony, which includes emotional, behavioural and, possibly, physiological components. Therefore, being able to quantify the synchrony could impact the way therapists adapt and maximise the efficacy of the interventions. However, current methods are based on the observational coding of behavior, which is time-consuming and usually only performed after the interaction is over. In this study we propose to apply Artificial Intelligence (AI) methods on physiological data in order to obtain a real time and objective quantification of synchrony. In particular, we introduce the Multi-Modal Synchrony dataset (M-MS), which includes 3 sources of information—electrocardiographic signals, video recordings and behavioral coding—to support the study of synchrony in ASD. As a first AI application, we are currently developing an unsupervised model to extract a multivariate embedding of the physiological data. The multivariate embedding has to be compared with the behavioral synchrony label to create a map of physiological and behavioural synchrony. The application of AI in the treatment of ASD may become a new asset for the clinical practice, especially if the possibility of providing real time feedback to the therapist is exploited.