WGAN-Based Robust Occluded Facial Expression Recognition

Abstract

Research on facial expression recognition (FER) technology can promote the development of theoretical and practical applications for our daily life. Currently, most of the related works on this technology are focused on un-occluded FER. However, in real life, facial expression images often have partial occlusion; therefore, the accurate recognition of occluded facial expression images is a topic that should be explored. In this paper, we proposed a novel Wasserstein generative adversarial network-based method to perform occluded FER. After complementing the face occlusion image with complex facial expression information, the recognition is achieved by learning the facial expression features of the images. This method consists of a generator G and two discriminators D1 and D2. The generator naturally complements occlusion in the expression image under the triple constraints of weighted reconstruction loss lwr, triplet loss lt, and adversarial loss la. We optimize the discriminator D1 to distinguish between real and fake by constructing an adversarial loss la between the generated complementing images, original un-occluded images, and small-scale-occluded images based on the Wasserstein distance. Finally, the FER is completed by introducing classification loss lc into D2. To verify the effectiveness of the proposed method, an experimental analysis was performed on the AffectNet and RAF-DB datasets. The visual occlusion complementing results, comparison of recognition rates of facial expression images with and without de-occlusion processing, and T-distributed stochastic neighbor embedding visual analysis of facial expression features all prove the effectiveness of the proposed method. The experimental results show that the proposed method is better than the existing state-of-the-art methods.