A Deep Learning Approach for Face Hallucination Guided by Facial Boundary Responses

Abstract

Face hallucination is a domain-specific super-resolution (SR) problem of learning a mapping between a low-resolution (LR) face image and its corresponding high-resolution (HR) image. Tremendous progress on deep learning has shown exciting potential for a variety of face hallucination tasks. However, most deep-learning-based methods are limited to handle facial appearance information without paying attention to facial structure priors. In this article, we propose an open source1 Boundary-aware Dual-branch Network (BDN) for face hallucination, which simultaneously extracts face features and estimates facial boundary responses from LR inputs, ultimately fusing them to reconstruct HR results. Specifically, we first upsample LR face images to HR feature maps, and then feed the upsampled HR features into a memory unit and an attention unit synchronously to obtain the refined features and predict facial boundary responses. Next, they are fed into a feature map fusion unit to combine facial appearance and structure information by a spatial attention mechanism. Moreover, we employ a series of stacked units to boost performance before recovering HR face images. Finally, a discriminative network is developed to improve visual quality by introducing adversarial learning strategy. Extensive experiments show that the proposed approach achieves superior face hallucination results against the state-of-the-art ones.