Facial Emotion Recognition using a Modified Deep Convolutional Neural Network Based on the Concatenation of XCEPTION and RESNET50 V2

Abstract

Facial emotion recognition has gained significant attention in modern years due to its wide applications in numerous fields, including human-computer interaction, market research and healthcare. This research focuses on improving facial emotion recognition accuracy by proposing a modified deep learning method based on the concatenation of Xception and ResNet50 architectures. The proposed approach aims to leverage the strengths of both Xception and ResNet50 networks to enhance facial expression representation and classification. Xception is known for its efficient feature extraction capabilities, while ResNet50 excels in capturing deeper and more complex patterns. By combining these architectures, the modified deep learning model can achieve higher emotion recognition accuracy. The research involves several stages. First, a large dataset of facial expressions is collected and preprocessed. The facial images are then fed into the modified deep-learning model, where feature extraction and classification occur. The model learns to recognize patterns and associations between facial expressions and specific emotions through a supervised learning process. Six distinct pre-trained DCNN models (ALEXNET, INCEPTIONV3, RESNET 50, VGG 16, XCEPTION and the concatenation of XCEPTION and RESNET50 V2) are used to validate the proposed system and with well-known datasets of FER2013, KDEF, CK+ JAFFE and with newly created custom Dataset-1 of 9K facial images. The proposed novel technique showed astounding accuracy, with a validation accuracy of 97.58% for a Softmax classifier, and it also recognized XCEPTION-RESNET V2 as the best network, with training and validation accuracy of 99.99% and 90%, respectively.