Advances on Coding and Transmission of Scalable Video and Multiview Video

The Advanced Video Coding (H.264/AVC) is the state-of-art video coding standard which has been developed by the Joint Video Team of ISO/IEC MPEG and ITU-T VCEG. It has been widely adopted in numerous products and services, such as TV broadcasting, video conferencing, mobile TV, and Blue-ray Disc. However, to support other application scenarios, for example, video delivery over heterogeneous networks, and enhanced user experiences in different aspects, advanced video representations of a scene are desired. From 2004 to 2008, JVT has developed video coding standards as the extensions of H.264/AVC: the scalable extension and the multiview extension, namely Scalable Video Coding (SVC) and Multiview Video Coding (MVC), respectively. SVC is designed to provide adaptation capability for heterogeneous network structures and different receiving devices with the help of temporal, spatial, and quality scalabilities. In addition, other potential scalabilities are investigated during the development of SVC. When a video sequence coded with SVC is delivered over an error-prone environment, it is challenging to achieve graceful quality degradation. Therefore, error resilient coding and error concealment techniques have been introduced for SVC. Some of the techniques are inherited from those for H.264/AVC, whereas some take advantages of the SVC features. The large amount of data needed to be processed by multiview applications is a heavy burden for both transmission and decoding. The MVC standard includes a number of new techniques for improved coding efficiency, reduced decoding complexity, and new functionalities. The system level integration of MVC is conceptually more challenging as the output of the decoder may contain any combination of the views with any temporal resolution level. Multiview video only enables rendering of a limited number of views. To achieve 3D rendering ability at any view angle or position, depth maps can be coded with texture video sequences. In this thesis, techniques for scalable and multiview video coding applications are proposed in the end to end systems based on SVC or MVC, including the design of standards, coding tools, encoder algorithms for high efficiency or improved error resilience, and decoder side error concealment. The contributions of this thesis are presented in Chapter 1 to Chapter 5. Each chapter includes a summary of a number of published papers by the author. These papers are attached to the thesis. Chapter 1 introduces H.264/AVC, focusing on the terminologies and the main techniques for SVC and MVC. An H.264/AVC baseline compliant high efficient encoding algorithm, which was based on hierarchical inter P picture structure, is proposed here. Chapter 2 and Chapter 3 give overviews for SVC and MVC, respectively. In Chapter 2, a color bit-depth scalable coding algorithm is proposed. It is a coding technology targeting future enhancement for SVC, and enabling storage or carriage of typical eight-bit video and high-bit video simultaneously with less bandwidth consumption. In Chapter 3, the presented techniques cover a wide range of MVC design, e.g., MVC bitstream structure, MVC transport, and MVC decoder resource management. The technologies contributed by the author, have been part of the MVC standard. Error concealment and error resilient methods for SVC and MVC, for an improved reconstructed video quality in an error-prone environment, are proposed in Chapter 4. In Chapter 5, the following coding techniques in specific multiview video coding applications are proposed: single-loop decoding for MVC, advanced asymmetric stereoscopic coding, and coding of 3D video content with depth maps. The above approaches summarize the recent advances in coding and transmission of SVC and MVC, contributed by the author. With the deployment of the SVC and MVC standards, the proposed techniques are expected to be widely used by industry, in this field, and are becoming important references for other relevant academic research