An Efficient Perceptual Video Compression Scheme Based on Deep Learning-Assisted Video Saliency and Just Noticeable Distortion

Abstract

Perceptual Video Compression (PVC) is one of the most promising approaches to enhancing compression efficiency. Saliency and Just Noticeable Distortion (JND) are the most common visual models in PVC. Most methods solely rely on one of them to construct PVC schemes, while a few methods attempt to integrate both. However, the latter are often too simplistic and coarse, failing to fully exploit them to reduce bitrate. To address this issue, we propose an efficient PVC scheme based on Deep Learning-Assisted Saliency and JND (SJ-PVC), which is implemented on the latest Versatile Video Coding (VVC). Specifically, we first design a Structurally Simplified Network (SS-Net) for video saliency prediction, which removes the redundancy of multi-scale models and maintains excellent accuracy. Then, an Adaptive Quantization Parameter (QP) selection algorithm based on Saliency and JND (SJAQP) is proposed. It can dynamically adjust QP offsets according to the characteristics of the encoding unit and effectively integrate the offset effects of saliency and JND. Finally, we design a Rate-Distortion Optimization based on Saliency and JND (SJRDO), which incorporates JND to obtain perceptual distortion terms and adjusts rate–distortion balance through saliency to achieve more rational bitrate allocation. This scheme fully considers the properties of both visual models and further eliminates visual redundancy. Extensive experiments demonstrate that, when maintaining subjective visual quality, SJ-PVC can save 22.87% of the bitrate compared to VVC while also shortening encoding time, significantly enhancing video coding efficiency.