End-To-End Emotion Recognition using Peripheral Physiological Signals

Abstract

The study of emotion recognition from physiological signals has seen a huge growth in recent decades. Studies initially used traditional machine learning classification to estimate either discrete emotions or combinations of arousal and valence. However, different feature engineering techniques such as Ensemble Empirical Mode Decomposition (EEMD) Analysis for electroencephalography (EEG) signals and statistical calculations for peripheral signals are used prior to machine learning. Also, several sensors needed to be worn by participants to predict the emotions. This study aims to investigate whether arousal and valence can be predicted from a single peripheral signal using deep learning. The Galvanic Skin Response(GSR), Respiration (RSP), Blood Volume Pulse (BVP) and Temperature (Temp) signals from the DEAP dataset are used. The signals are downsampled to approximately three hertz (Hz) and input to a convolutional network (CNN) to predict arousal and valence. GSR, RSP and BVP had similar F1 and accuracy results. BVP had an F1 result of 0.673 and 0.632 and accuracies of 63.5% and 61.1% respectively for arousal and valence. RSP's F1 results were 0.677 and 0.669 and accuracies were 61.3% and 64.2% for arousal and valence respectively. GSR had F1 results of 0.699 and 0.663 and accuracies of 62.5% and 60.2% respectively for arousal and valence. Using raw signals and examining the peripheral signals individually, we were able to identify which sensors showed the best potential for further research to bring emotion classification into a real-world scenario using non-invasive sensors.