A Global-Motion-Compensation Based Video Coding Scheme and Its Application to Video Resolution Enhancement

全域移動預測 (global motion estimation) 在許多應用中被廣泛使用，而全域移動補償 (global motion compensation) 的概念在過去十年中蓬勃發展，至今已達成熟並運用於視訊編碼標準。另一方面，為提高視訊影像之解析度 (video resolution enhancement)，除了硬體製造技術改良求新之外，運用多張低解析度影像達到高解析度影像重建之技術，已被驗證其高效能並廣泛地使用在各種領域中。 本論文針對MPEG-4全域移動補償工具提出技術改良，以期在更複雜的影片場景中能有更高的利用效能。我們也將全域移動補償的觀念引至最新的視訊編碼標準，H.264/AVC。由實驗結果證實，對於含有明顯相機移動 (camera motion) 的影片，採用我們所提出的技術與先前的標準相比，其壓縮結果更為有效率。 此外，本論文提出一視訊解析度提升之架構。此架構考慮了常見的相機移動和前後景動作之不一致 (inconsistent moving foreground)，在提升視訊影像銳利度的同時並能防止雜訊的產生。經由主觀實驗測試證實本論文所提出的方法擊敗了常見的畫素內插法 (pixel interpolation) 及前人所提出的文獻探討。 最後，我們將全域移動補償及視訊解析度之提升結合為一項新的應用。在此應用中，運算最為複雜耗時的全域移動預測工作將被移至編碼端 (encoder)，所產生的移動參數 (motion parameter) 將由視訊串流傳遞至解碼端 (decoder)。如此一來，解碼端只需進行簡單的畫素註冊動作 (pixel registration)，對於有速度考量之應用便可在有限的時間內完成。實驗結果驗證了我們的方法只需引入極少的額外頻寬，即可有效率地達到解析度之提升。Global motion estimation helps in many applications. The concept of global motion compensation (GMC) has been developed in the last decade, and reaches maturity for being applied to video coding recently. On the other hand, super-resolution (SR) image reconstruction is widely used to build a high spatial resolution image through referencing a series of low-resolution (LR) images of the same scene. In this thesis, we propose some refinements to the global motion compensation in MPEG-4 Advanced Simple Profile (ASP) for higher utilization in complex scenes. We also introduce the GMC concept into the latest video coding standard, H.264/AVC. Experiment results show that by applying our scheme, videos with apparent camera motion, i.e., pan, rotate, or zoom, are coded in a more efficient way. On the other hand, a scheme of video resolution enhancement is proposed. It considers common camera motion and precise outlier removal which enhance the video sharpness while avoid unfavorable noise interference at the same time. Through subjective quality test, our proposed scheme outperforms the simple interpolation method and previous SR approaches. Finally, a framework to combine GMC and SR reconstruction for video decoding applications is designed. In the proposal, most computation-intensive task is shifted to an offline encoding process. All GMC parameters are generated and embedded in the video bitstreams. Then, the major task for the decoder is simply doing the registration, whose computation is acceptable in time-constrained applications. In our experiments, the proposal can produce a high quality video up to 4 times large in each spatial dimension while just introducing an unnoticeable bitrate increase.CHAPTER 1 INTRODUCTION 1 1.1 GLOBAL MOTION 1 1.2 WHY IS GLOBAL MOTION ESTIMATION (GME) IMPORTANT? 2 1.2.1 GME v.s. Video Coding 2 1.2.2 GME v.s. Image/Video Resolution Enhancement 4 1.2.3 Real-time SR Decoding Applications 6 1.3 CONTRIBUTIONS 6 1.4 THESIS ORGANIZATION 7 CHAPTER 2 RELATED WORK 9 2.1 PRIOR WORKS ON GLOBAL MOTION ESTIMATION 9 2.2 PRIOR WORKS ON GME IN VIDEO CODING 10 2.3 PRIOR WORKS ON SUPER RESOLUTION RECONSTRUCTION 14 CHAPTER 3 GME FOR VIDEO RECONSTRUCTION 21 3.1 INTRODUCTION 21 3.2 PROPOSED REFINEMENT SCHEME 23 3.3 EXPERIMENTAL RESULTS 26 3.4 SUMMARY 30 CHAPTER 4 GMC-BASED VIDEO CODING 31 4.1 INTRODUCTION TO APPLYING GMC IN H.264/AVC 31 4.2 MODIFIED MOTION ESTIMATION 32 4.2.1 Modified Initial Translational Estimate 32 4.2.2 Outliers Information Utilization 32 4.3 GMC IN H.264/AVC 33 4.3.1 Macroblock-based Motion Compensation 33 4.3.2 Mode Decision and Encoding 34 4.4 EXPERIMENTS 34 4.5 SUMMARY 36 CHAPTER 5 IMAGE/VIDEO RESOLUTION ENHANCEMENT 37 5.1 INTRODUCTION 37 5.2 THE PROPOSED SUPER RESOLUTION IMAGE RECONSTRUCTION 38 5.2.1 Global Motion Estimation 39 5.2.2 Image Registration 39 5.2.3 Outlier Removal 41 5.2.4 Post Processing for Image Quality Enhancement 42 5.3 SR VIDEO RECONSTRUCTION 42 5.4 EXPERIMENTS 43 5.5 SUMMARY 47 CHAPTER 6 GMC-BASED SR DECODER 49 6.1 INTRODUCTION 49 6.2 IMPLEMENTATION DETAILS 50 6.3 EXPERIMENTS 50 6.4 SUMMARY 51 CHAPTER 7 CONCLUSIONS AND FUTURE WORK 53 7.1 CONCLUSIONS 53 7.2 FUTURE WORK 54 BIBLIOGRAPHY …………………………………………………………………55 APPENDIX A ………………………………………………………………………5