Free-Viewpoint 3DTV : view interpolation, coding, and streaming

Free-ViewPoint (FVP) interpolation allows creating a new view between the existing reference views. Applied to 3D multi-view video sequences, it leads to two important applications: (1) FVP service provided to the user, which enables the possibility to interactively select the viewing point of the scene; (2) improved compression of multi-view video sequences by using view prediction for inter-view coding. In this chapter, the authors provide an overview of the essential steps for 3D free-view video communication, which consists of the free-viewpoint interpolation techniques, a concept for free-view coding and a scalable free-view video streaming architecture. For facilitating free-view to the user, the chapter introduces the free-viewpoint interpolation techniques and the concept of warping. The authors assume that 3D video is represented by texture and depth images available for each view. Therefore it is possible to apply Depth Image Based Rendering (DIBR), which uses the depth signal as a important cue for geometry information and 3D reconstruction. Authors analyze the involved interpolation problems, such as cracks, ghost contours and disocclusions, which arise from an FVP interpolation and propose several solutions to improve the image quality of the synthesized view. Afterwards, they present a standard approach to FVP rendering used currently by the research community and our FVP interpolation. Additionally, authors show the use of FVP rendering for the multi-view coding and streaming and discuss the gains and trade-offs of it. At the end of the chapter are the state-of-the-art achievements and challenges of FVP rendering and a vision concerning the development of free-viewpoint services