Fast Intra- and Inter-Prediction Mode Decision in H.264 Advanced Video Coding

Abstract- H.264/AVC, the latest video coding standard, achieves better video compression rates since it supports new features such as a large number of intra- and inter-prediction candidate modes. H.264/AVC adopts rate-distortion optimization (RDO) technique to obtain the best intra- and inter-prediction, while maximizing visual quality and minimizing the required bit rate. However, full RD cost calculation for all intra-prediction modes and exhaustive searches for optimal motion vectors for all block sizes increase computational complexity considerably with the number of prediction modes allowed. In order to reduce the complexity, we propose an enhanced fast intra-prediction mode selection strategy based on the statistical properties of references pixels and a fast inter-prediction mode decision approach based on the correlation of motion vectors and split/merge procedure. The objective is to reduce the number of candidate modes in intra- and inter-prediction while maintaining the coding efficiency. Experimental results show that our algorithm, compared to the RDO and other fast algorithms, reduces the total encoding time with negligible loss in PSNR and a slightly increased bitrate. Keywords- H.264/AVC, RDO, intra-prediction, inter-prediction. 1