A dual-network based super-resolution for compressed high definition video

Abstract

Convolutional neural network (CNN) based super-resolution (SR) has achieved superior performance compared with traditional methods for uncompressed images/videos, but its performance degenerates dramatically for compressed content especially at low bit-rate scenario due to the mixture distortions during sampling and compressing. This is critical because images/videos are always compressed with degraded quality in practical scenarios. In this paper, we propose a novel dual-network structure to improve the CNN-based SR performance for compressed high definition video especially at low bit-rate. To alleviate the impact of compression, an enhancement network is proposed to remove the compression artifacts which is located ahead of the SR network. The two networks, enhancement network and SR network, are optimized stepwise for different tasks of compression artifact reduction and SR respectively. Moreover, an improved geometric self-ensemble strategy is proposed to further improve the SR performance. Extensive experimental results demonstrate that the dual-network scheme can significantly improve the quality of super-resolved images/videos compared with those reconstructed from single SR network for compressed content. It achieves around 31.5% bit-rate saving for 4 K video compression compared with HEVC when applying the proposed method in a SR-based video coding framework, which proves the potential of our method in practical scenarios, e.g., video coding and SR.