Image Super-Resolution Using Knowledge Distillation

Abstract

The significant improvements in image super-resolution (SR) in recent years is majorly resulted from the use of deeper and deeper convolutional neural networks (CNN). However, both computational time and memory consumption simultaneously increase with the utilization of very deep CNN models, posing challenges to deploy SR models in realtime on computationally limited devices. In this work, we propose a novel strategy that uses a teacher-student network to improve the image SR performance. The training of a small but efficient student network is guided by a deep and powerful teacher network. We have evaluated the performance using different ways of knowledge distillation. Through the validations on four datasets, the proposed method significantly improves the SR performance of a student network without changing its structure. This means that the computational time and the memory consumption do not increase during the testing stage while the SR performance is significantly improved.