Integrated Fast Mode Decision Algorithm and SSIM-Based Rate-Distortion Optimization for H.264 Encoder

H.264標準化的成功意味著下一代視訊編碼標準的編碼工具將變得更加複雜且需要大量的運算以因應我們朝高畫質影像邁進的發展趨勢。於是，為了滿足大量的消費性電子和多媒體通信應用的即時性需求，開發提高先進編碼工具之計算效率的演算法是相當重要的。另一方面，由於視訊品質的優劣，最終依然由人的視覺感知所決定，所以我們堅定地認為，在設計下一代視訊編碼系統時，將人類視覺特性列入演算法設計的考量是必要的。本論文主要由兩部分組成：分別是整合性快速模式決策與基於結構相似性之位元率-失真 (Rate-Distortion, 簡稱R-D)優化演算法。第一部分中，我們針對H.264模式決策階層中之三個不同層級分別提出其快速模式決策演算法，分別是基於變異數之宏塊模式決策、基於濾波器之預測模式決策之強化、與依據R-D特徵的選擇性畫面內模式決策。它們的整合方式也是研究重點之一，並進一步分別提出對畫面內預測幀編碼與畫面間預測幀編碼之整合性快速演算法。整合性演算法大量降低計算複雜度卻不造成明顯的R-D表現損失。實驗結果亦顯示提出之演算法的優越性。第二部分中，我們則基於結構相似性 (SSIM) 制定了一個R-D優化的架構以應用於H.264的模式決策過程，並提出可適用於此架構之預測性拉格朗日乘數選擇方法。為滿足不同應用之需求，不同計算複雜度之預測方式分別提出並討論。而在以SSIM衡量之影像品質相同下，我們所提出的方法可達到約5 ％ -10 ％的位元率減少。由主觀的視覺評估可發現，在相同位元率的限制下，相較於傳統基於MSE優化之H.264編碼器，所提出的方法可保留更多的細節並且產生較少的區塊效應，進而得到較佳的影像品質。The success of H.264 standardization implies that the video coding tools of the next-generation video coding standard, for example, H.265, will become more complicated and require extensive computations for high quality video. To satisfy the real-time requirements of many consumer electronic and multimedia communication applications, it is absolutely necessary to enhance the computational efficiency of such advanced coding tools. On the other hand, because the video quality is ultimately judged by human eyes, we strongly believe that the characteristics of human visual system must be taken into account in the design of the next-generation video coding system. Motivated by these requirements of next-generation video coding, this thesis targets the development of algorithm for 1) integrated fast mode decision algorithm and 2) structural similarity based rate distortion optimization.n the first part, three fast intra mode decision algorithms for different stages in the mode decision hierarchy of H.264 are proposed, which are variance-based MB mode decision, improved filter-based prediction mode decision, and an R-D characteristic based selective intra mode decision. Their integration is also investigated and we propose integrated fast algorithms for intra-frame coding and inter-frame coding, respectively. The integrated algorithms achieve high complexity reduction without introducing noticeable R-D performance loss. The experimental results are provided to show the superiority of the proposed algorithms.n the second part, we develop a rate-distortion optimization framework based on structural similarity for the mode decision process in H.264, and propose a predictive Lagrangian multiplier selection method for the proposed framework. To estimate the Lagrangian multiplier, approaches with different computational overhead are presented to meet the requirement of different target applications. The proposed method achieves about 5%-10% bit rate reduction with same quality in terms of SSIM index. From the subjective evaluation, the proposed method preserves more detail and introduces less block artifact than the MSE-based H.264 encoder with the same bit-rate constraint.致謝 i文摘要 iiBSTRACT ivONTENTS viIST OF FIGURES xIST OF TABLES xivhapter 1 Introduction 1.1 Video Coding Standards 2.2 Research Contribution 4.3 Thesis Organization 5hapter 2 Preliminary 7.1 Overview of H.264 Encoder 7.2 Transform and Quantization 9.3 Entropy Coding 10.4 Intra Prediction 10.5 Motion Estimation 12.6 Rate-Distortion Optimized Mode Decision 15.7 Performance Metric and Structural Similarity Index 16hapter 3 Related Work 19.1 Categories of Fast Mode Decision Algorithms for H.264 19.2 Fast Prediction Mode Decision for Intra Prediction 20.2.1 Literature Survey 20.2.2 A Fast Mode Decision Algorithm and Its VLSI Architecture for H.264/AVC Intra Prediction [32] 22.3 Fast Selective Intra Mode Decision for Inter-frame Coding 24.3.1 Literature Survey 24.3.2 Fast Coding Mode Selection with Rate-Distortion Optimization for MPEG-4 Part-10 AVC/H.264 [56] 25.4 Lagrange Multiplier Selection for Rate Distortion Optimization Framework 26.5 Perceptual Based Video Coding 29hapter 4 Integrated Fast Mode Decision Algorithm 31.1 Variance-base MB Mode Decision 31.1.1 Motivation and Analysis 31.1.2 Proposed Method 33.1.3 Experimental Results 34.2 Improved Prediction Mode Decision 35.2.1 Motivation and Analysis 35.2.2 Proposed Method 38.2.3 Experimental Results 39.3 R-D Characteristic-based Selective Intra Mode Decision 40.3.1 Motivation and Analysis 40.3.2 Proposed Method 42.3.3 Experimental Results 44.4 Integration for Intra-frame Coding 48.4.1 Proposed Algorithm 48.4.2 Experimental Results 49.5 Integration for Inter-frame Coding 52.5.1 Proposed Algorithm 52.5.2 Experimental Result 54hapter 5 Structural Similarity Based Rate Distortion Optimization 57.1 SSIM-based Rate Distortion Optimization 57.1.1 Motivation 57.1.2 Rate Distortion Optimized Mode Decision Based on SSIM 58.2 Predictive Lagrangian Multiplier Selection 59.2.1 Observation and Analysis 59.2.2 Framework of the Proposed Algorithm 62.2.3 Gradient Descent Based Approach 63.2.4 Slope Approximation Based Approach 64.2.5 Model-based Low Complexity Mode 65.2.6 Periodic Refreshment 66.3 Experimental Results 67.3.1 Improvement on the prediction mode decision of I4MB 68.3.2 Improvement on the MB mode decision of an intra MB 71.3.3 Evaluation of the Estimated λ 75.3.4 Issue on periodic refreshment 77.3.5 Model-based approach for video conferencing application 78.3.6 Complexity analysis 79.3.7 Subjective Evaluation 80hapter 6 Conclusion 87EFERENCE 8