Multi-modal stress recognition using temporal convolution and recurrent network with positional embedding

Abstract

Chronic stress causes cancer, cardiovascular disease, depression, and diabetes, therefore, it is profoundly harmful to physiologic and psychological health. Various works have examined ways to identify, prevent, and manage people's stress conditions by using deep learning techniques. The 2nd Multimodal Sentiment Analysis Challenge (MuSe 2021) provides a testing bed for recognizing human emotion in stressed dispositions. In this study, we present our proposal to the Muse-Stress sub-challenge of MuSe 2021. There are several modalities including frontal frame sequence, audio signals, and transcripts. Our model uses temporal convolution and recurrent network with positional embedding. As result, our model achieved a concordance correlation coefficient of 0.5095, which is the average of valence and arousal. Moreover, we ranked 3rd in this competition under the team name CNU_SCLab.