Integrating Deep Metric Learning, Semi Supervised Learning, and Domain Adaptation for Cross-Dataset EEG-Based Emotion Recognition

Abstract

The variability of EEG signals produced by different trial conditions and different devices presents significant challenges in developing practical EEG-based emotion recognition systems. Much of the research on developing a generalizable EEG emotion recognition approach focuses on cross-subject and cross-session contexts. Although current cross-subject methods yield satisfactory outcomes on certain EEG emotion datasets, their effectiveness diminishes in cross-dataset scenarios. Additionally, the performance of existing cross-dataset methods remains inferior compared to methods that are trained and evaluated within the same dataset. To address these challenges, inspired by the effective application of deep metric learning (DML) in zero-shot and few-shot learning tasks, this paper introduces a cross-dataset emotion recognition method. The proposed approach integrates DML, domain-specific batch normalization (DSBN), shared batch normalization statistics, and adversarial learning. Specifically, our method extracts cross-domain features from the input signals using DSBN and shared batch normalization statistics. Then, the proposed DML loss minimizes intra-class variations of EEG features across different subjects and domains, while maximizing differences between different classes. Moreover, it captures the semantic order of emotions in the learned embedding. To further improve the generalization of the feature encoder, we employ adversarial learning with domain and subject discriminators. We evaluate our method on six cross-dataset scenarios. The results show that it consistently outperforms peer methods across the scenarios. For example, our method achieves an accuracy of 63.49% on SEED to SEED-IV, improving the state-of-the-art result by 2.25%.