Facial expression recognition based on ensemble of mulitple CNNs

Abstract

Automatic recognition of facial expression is an important task in many applications such as face recognition and animation, human-computer interface and online/remote education. It is still challenging due to variations of expression, background and position. In this paper, we propose a method for facial expression recognition based on ensemble of multiple Convolutional Neural Networks (CNNs). First, the face region is extracted by a face detector from the pre-processed image. Second, five key points are detected for each image and the face images are aligned by two eye center points. Third, the face image is cropped into local eye and mouth regions, and three CNNs are trained for the whole face, eye and mouth regions, individually. Finally, the classification is made by ensemble of the outputs of three CNNs. Experiments were carried for recognition of six facial expressions on the Extended Cohn-Kanade database (CK+). The results and comparison show the proposed algorithm yields performance improvements for facial expression recognition.