Improved rate-distortion optimization algorithms for hevc lossless coding

Abstract

To avoid distortion, the quantization is not implemented on residues for lossless mode in HEVC. As a result, the conventional lambda model in Rate-Distortion Optimization (RDO), where lambda is related to the quantization parameter (QP), is unreasonable for lossless coding. This paper first demonstrates the role that lambda value plays in the rough RDO of HEVC lossless coding, and a Simulated Annealing algorithm based approach is proposed to obtain the most appropriate lambda for each largest coding unit. Considering the computational complexity, the other simplified method using least square errors prediction is proposed to improve the rough RDO process. Experimental results reveal that on top of the lossless coding mode, the improved method offers the performance with a 1.0%, 1.3% and 1.1% bit-rate reduction on average for Random-access Main, Low-delay B Main and Low-delay P Main configurations, respectively, while brings negligible increases of computational complexity in the encoder.