Joint Learning of Super-Resolution and Perceptual Image Enhancement for Single Image

Abstract

Super resolution (SR) and Perceptual Image Enhancement (PIE) are gaining more and more interests in digital image processing and have been studied independently in the past decades. Although plenty of state-of-the-art researches have demonstrated great improvement in SR problem, they neglect practical requirements in real-world application. In practice, these two tasks are always mixed and combined to obtain a high-resolution enhanced (HRE) image with high quality from a low-resolution original image (LRO) with low quality. In this paper, we propose a joint SR-PIE learning framework called Deep SR-PIE, which comprises Multi-scale Backward Fusion Network (MBFNet), Perceptual Enhancement Network (PENet) and Dual-Path Unsampling Network (DUNet). MBFNet network is responsible for deep feature representation for further image reconstruction and perceptual enhancement, and PENet seeks the optimal local transformation to recover perceptual loss (color, tone, exposure and so on). DUNet works in different scales and exchanges each other to complement more details during upsampling. In our experiments, a real-world dataset is released to facilitate the development of joint learning for SR and PIE. Then, a thorough ablation study is provided to better understand the superiority of our method. Finally, extensive experiments suggest that the proposed method performs favorably against the state-of-the-arts in terms of visual quality, PSRN, SSIM, model size and inference time. By virtue of splitting operation and inverse residual blocks, as a lightweight deep neural network, our model is compatible with low-computation device.