Finely-Grained Real-Time Facial Emotion Recognition Towards Neural Network

Abstract

Over the past assorted decades, one of the most agile spheres of research is the emotion recognition. The grounds of this study is to come forward with finely grained real-time learning model of emotion recognition comprising of phases like feature extraction, subset feature and emotion classifier. Haar Cascade technique is being used for identifying the input figure to identify the characteristic values. This work also aims to classify human emotions like fear, neutral, angry, surprise, happy and sad by using two classifiers; Long Short Term Memory (LSTM) and Convolutional Neural Network (CNN) by deploying a finely grained real time emotion recognition algorithm using virtual markers. Initially, Haar Cascade was used for eyes and face discernment, then Neighborhood Difference Features (NDF) were extracted and virtual markers are being positioned on designated places on identified face using VGG16 approach. The facets are validated using cross-validation and forwarded to CNN and LSTM classifiers. Experimental results of loss function, confusion matrix, classification report of training and testing had proved that the model proposed give consistent output with the real time facial expressions.