Ultra-resolving face images by discriminative generative networks

Abstract

Conventional face super-resolution methods, also known as face hallucination, are limited up to 2∼4× scaling factors where 4 ∼ 16 additional pixels are estimated for each given pixel. Besides, they become very fragile when the input low-resolution image size is too small that only little information is available in the input image. To address these shortcomings, we present a discriminative generative network that can ultra-resolve a very low resolution face image of size 16 × 16 pixels to its 8× larger version by reconstructing 64 pixels from a single pixel. We introduce a pixel-wise l2 regularization term to the generative model and exploit the feedback of the discriminative network to make the upsampled face images more similar to real ones. In our framework, the discriminative network learns the essential constituent parts of the faces and the generative network blends these parts in the most accurate fashion to the input image. Since only frontal and ordinary aligned images are used in training, our method can ultra-resolve a wide range of very low-resolution images directly regardless of pose and facial expression variations. Our extensive experimental evaluations demonstrate that the presented ultra-resolution by discriminative generative networks (URDGN) achieves more appealing results than the state-of-the-art.