Three-dimensional propagation algorithm for depth image-based rendering

Depth image based rendering (DIBR) is the process of synthesizing new ``virtual'' views from a set of ``real'' views which include color and depth images. Because of its photorealism and less stringent computational requirement, DIBR has many applications such as 3D TV, remote reality, and video conferencing, and has become one of the hot research areas in visual computing in recent years. Since the general purpose graphics processing unit (GPGPU) is an ideal computing platform for image rendering, we consciously develop a novel and necessary image processing algorithm suitable for GPGPU by exploiting massive parallelism. The proposed 3D propagation algorithm for DIBR combines images from multiple color and depth cameras at arbitrary positions in 3D space and efficiently renders novel images at arbitrary virtual views by propagating all available depth information from depth cameras to color cameras, and then all available depth and color information from the color cameras to the virtual views. Furthermore, we consider the case when only low resolution depth images are obtained. A novel depth filling and enhancement technique for enhancing depth image quality using high resolution color images is proposed and significantly improves the rendering quality. Finally, the paper also describes the abundant but irregular parallelism of our algorithm and outlines a mapping onto massively parallel architectures such as GPGPUs