Face image illumination processing based on GAN with dual triplet loss

Abstract

It is generally known that the illumination could seriously affect the performance of face analysis algorithms. Moreover, in most practical applications, the illumination is usually uncontrolled. A number of methods have been put forward to tackle the problem of illumination variations in face images, but they always only work on facial region and need to segment faces in advance. Furthermore, many illumination processing methods only demonstrate on grayscale images and require strict alignment of face images, resulting in limited applications in the real world. In this paper, we propose a face image illumination processing method based on the Generative Adversarial Network (GAN) with dual triplet loss. Through considering the inter-domain similarity and intra-domain difference between the generated images and the real images, we put forward the dual triplet loss. At the same time, we introduce the self-similarity constraint of the images in the target illumination field. Experiments on the CMU Multi-PIE face datasets demonstrate that the proposed method preserve the facial details well when relighting. The experiment of 3D face reconstruction also verifies the effectiveness of the proposed method.