Enhancing Emotion Detection from EEG Signals by Extracting Spatiotemporal Features Using ConvLSTM Networks

Abstract

Emotion detection is a key factor in good interactions between people in daily routines. Human-Computer Interaction (HCI)-based intelligent systems are required to distinguish the emotional states of subjects. Electroencephalography (EEG) is a more reliable and cost-effective method for measuring brain activity. It makes it possible to comprehend human emotional and cognitive processes. However, emotions are dynamic and include complex relationships among different brain regions throughout time. Different Deep Learning (DL)-based automated HCI systems are able to detect emotions from EEG signals, but the challenge is that little research has been conducted on spatiotemporal feature-based detection. Most of the research has either used Convolutional Neural Networks (CNNs), a spatial model, or Long Short-Term Memory (LSTM), a temporal model, which is unable to detect spatiotemporal information at the same time. This research has introduced a spatiotemporal DL model that integrates CNN and LSTM models to obtain spatiotemporal features from EEG signals. Features are extracted in two ways, i.e., automatic and manual feature extraction. CNN is treated as an automated feature extractor, while manual features are extracted using spatial-temporal and connective feature extractors. An LSTM model is trained on the extracted features, which further extracts temporal features from the EEG data. The proposed methods are evaluated on two benchmark datasets, SJTU Emotion EEG Dataset (SEED) and Database for Emotion Analysis using Physiological Signals (DEAP). The experimental results show enhanced performance compared to recent literature in terms of accuracy, achieving 97.98% and 75.85% in the SEED and DEAP datasets, respectively.