Facial expressions recognition with multi-region divided attention networks for smart education cloud applications

Abstract

In recent years, the electronic devices and wireless network are seen everywhere, generating a massive amount of online surveillance video data that can be applied to recognize facial expressions to sustain the smart education cloud deployment. However, research highlights of existing Facial Expression Recognition (FER) methods mainly focus on the global or local facial expression separately, but pay less attention to the co-operation relationships among them. To relieve the problem, this paper has proposed a Multi-region Attention Transformation Framework (MATF) to blend the global and local facial details for expression recognition. The proposed framework is structured with a multi-region attention transformation network and a pseudo label generation module. The former is used to fuse the rough global feature and local detail information, which divides the original facial image into multi-region sub-blocks for multi-region expression association. Specifically, it includes the facial local segmentation network, the attention transformation network and the feature weight allocation mechanism for facial expression feature extraction. The pseudo label generation strategy is proposed to enhance the performance of multi-region facial expression integration with a semi-supervised way. Further, a unified training strategy is exploited to optimize the proposed framework to ensure a high FER accuracy. Experiments have been conducted on several public FER datasets (RAF-DB, FERPlus, CAER-S) and results indicate that the proposed method outperforms existing algorithms by 3%–8% in accuracy. Based on the multi-region divided attention network learned by the proposed framework, the algorithm for recognizing the expressions can achieve a time complexity as low as O(n), which can be deployed in the mobile electronic devices like the FaceReader series smart facial expression analysis sever. In addition, our facial expression recognition algorithm can be also deployed into the smart education cloud for E-learning.