Temporal Facial Features for Depression Screening

Abstract

Depression is a common and debilitating mental illness. Given the shortage of mental health professionals, there are delays in depression detection. Interviews conducted by virtual agents could expedite depression screenings. While the interview audio and transcript have received more attention, facial features offer an attractive privacy-preserving screening modality. Thus, we conduct a comprehensive comparative evaluation of the effectiveness of temporal facial features to screen for depression. We extract time series of eye gaze, landmark, and action unit features from video responses to 15 clinical interview questions. We input them into CNN, LSTM, and recurrent convolutional neural network (RCNN) models. An extra attention layer proved critical for CNN and LSTM performance. For a general wellbeing question, eye gaze features screened for depression with an F1 of 0.81. Our study informs the use of temporal facial features in future digital mental illness screening technologies.