I Got Your Emotion: Emotion Preserving Face De-identification Using Injection-Based Generative Adversarial Networks

Abstract

Traditional visual information concealing methods involve blurring or pixelating the face, which can obscure emotional expression and lead to misunderstandings and misinterpretations. This research proposes the very first end-to-end emotion-preserving de-identification method that employs an end-to-end Generative Adversarial Network (GAN) for producing natural-looking de-identified images. The proposed method is based on the StyleGAN architecture to generate a synthetic face image dataset for proxy face images, followed by using the DeepFace model to classify the gender of the target image and selecting a representative image with the same gender for face swapping. An enhanced SimSwap framework is proposed to improve the emotion preservation quality of the de-identification. A new loss function is introduced specifically to preserve emotional expressions. The deep face model is used to classify the original image’s emotional expression, and then the same model is used to recognize the emotional expression of the de-identified swapped image. Emotional expressions are explicitly preserved during face-swapping by minimizing attribute preservation loss. The proposed method outperformed the most recent face de-identification method in terms of accuracy and emotion-preserving capabilities.