A Novel Face Frontalization Method by Seamlessly Integrating Landmark Detection and Decision Forest into Generative Adversarial Network (GAN)

Abstract

In real-world scenarios, posture variation and low-quality image resolution are two well-known factors that compromise the accuracy and reliability of face recognition system. These challenges can be overcome using various methods, including Generative Adversarial Networks (GANs). Despite this, concerns over the accuracy and reliability of GAN methods are increasing as the facial recognition market expands rapidly. The existing framework such as Two-Pathway GAN (TP-GAN) method has demonstrated that it is superior to numerous GAN methods that provide better face-texture details due to its unique deep neural network structure that allows it to perceive local details and global structure in a supervised manner. TP-GAN overcomes some of the obstacle associated with face frontalization tasks through the use of landmark detection and synthesis functions, but it remains challenging to achieve the desired performance across a wide range of datasets. To address the inherent limitations of TP-GAN, we propose a novel face frontalization method (NFF) combining landmark detection, decision forests, and data augmentation. NFF provides 2D landmark detection to integrate global structure with local details of the generator model so that more accurate facial feature representations and robust feature extractions can be achieved. NFF enhances the stability of the discriminator model over time by integrating decision forest capabilities into the TP-GAN discriminator core architecture that allows us to perform a wide range of facial pose tasks. Moreover, NFF uses data augmentation techniques to maximize training data by generating completely new synthetic data from existing data. Our evaluations are based on the Multi-PIE, FEI, and CAS-PEAL datasets. NFF results indicate that TP-GAN performance can be significantly enhanced by resolving the challenges described above, leading to high quality visualizations and rank-1 face identification.