Correlation-Based Attention Weights Mechanism in Multimodal Depression Detection: A Two-Stage Approach

Abstract

Globally, the COVID-19 pandemic has aggravated the problem of depression. In the literature on depression detection, predictive capacity of multi-modal biomarkers like speech, facial and language characteristics were combined to improve the performance of Machine Learning (ML) models. The existing techniques have the limitation of not using the feature based weighting on multimodal data items in supervised learning models. The aim of the study is to use correlation based weighting mechanism between the feature and label in supervised learning to build an automatic multimodal depression detection system. We constructed a dataset of audio-visual recordings that consists of depressed and non-depressed subjects, to address this limitation by proposing a novel two-stage multimodal method for classification. The first stage attention weight is found using the Pearson correlation coefficients. Then these attention weights are combined with the multimodal feature vector, to use in the second stage. The resultant feature vector is experimented with three state-of-the-art classifiers i.e. Support Vector Machine (SVM), Decision Tree (DT) and K-Nearest Neighbour (KNN) classifiers, for predictions. Overall performance measures of proposed model are: accuracy 87.3%, precision 83.3%, recall 77%, and f1-score 80% which showed its effectiveness. Also outcomes of this proposed model were compared with the outcomes of existing state-of-the-art methods, such as baseline base classifiers and feature-level fusion method. Further, the proposed model is also evaluated applying to a widely used benchmarking dataset i.e. Distress Analysis Interview Corpus (DAIC) in depression detection. Results showed that combination of heterogeneous classifiers improves performance when classifying depressed and non-depressed subjects.