HEVC Encoder Optimization Based on a New RD Model and Pre-Enoding

Abstract — In this paper, a new Rate Distortion (RD) model based on the content measured by the sum of absolute transformed differences (SATD) is developed. Based on this advanced RD model we propose an encoding optimization method for HEVC codec. First, we perform pre-compression with multiple quantization parameters (QPs) on selected coding modes (SKIP, Inter8x8, Intra8x8) for the current frame and get predicted bitrates and distortions. Second, we use this predicted R and D data to get our RD model's parameters by using the Least Squares method. Finally, with this RD model, a pair of λ and QP values is estimated for the actual encoding process and is used to encode the current frame. We applied our method to HM3517, and the results show an average of-2.20%,-0.86%, and-1.88 % BD-rate decrease for the Y, U and V components. The maximum decrease in the Y component reaches-6.2 % BD-Rate. I