An Improved Facial Expression Recognition using CNN-BiLSTM with Attention Mechanism

Abstract

In the recent years, Facial Expression Recognition is one of the hot research topics among the researchers and experts in the field of Computer Vision and Human Computer Interaction. Traditional deep learning models have found it difficult to process images that has occlusion, illumination and pose dimensional properties, and also imbalances of various datasets has led to large distinction in recognition rates, slow speed of convergence and low accuracy. In this paper, we propose a hybrid Convolution Neural Networks-Bidirectional Long Short Term Memory along with point multiplication attention mechanism and Linear Discriminant analysis is incorporated to tackle aforementioned non-frontal image properties with the help of Median Filter and Global Contrast Normalization in data preprocessing. Following this, DenseNet and Softmax is used for reconstruction of images by enhancing feature maps with essential information for classifying the images in the undertaken input datasets i.e. FER2013 and CK+. The proposed model is compared with other traditional models such as CNN-LSTM, DSCNN-LSTM, CNN-BiLSTM and ACNN-LSTM in terms of accuracy, precision, recall and F1 score. The proposed network model achieved highest accuracy in classifying the facial images on FER2013 dataset with 95.12% accuracy which is 3.1% higher than CNN-LSTM, 2.7% higher than DSCNN-LSTM, 2% higher than CNN-BiLSTM and 3.7% higher than ACNN-LSTM network models, and the proposed model has achieved 98.98% of accuracy with CK+ in classifying the images which is 5.1% higher than CNN-LSTM, 5.7% higher than DSCNN-LSTM, 3.3% higher than CNN-BiLSTM and 6.9% higher than ACNN-LSTM network models in facial expression recognition.