A 3D Modeling and Free-View Generation System Using Environmental Stereo Cameras

ABSTRACT: We propose a 3D video system that uses environmental stereo cameras to display a target object from an arbitrary viewpoint. This system is composed of the following stages: image acquisition, foreground segmentation, depth field estimation, 3D modeling from depth and shape information, and arbitrary view rendering. To create 3D models from captured 2D image pairs, a real-time segmentation algorithm, a fast depth reconstruction algorithm, and a simple and ef-ficient shape reconstruction method were developed. For viewpoint generation, the 3D surface model is rotated toward the desired place and orientation, and the texture data extracted from the original cam-era is projected onto this surface. Finally, a real-time system that demonstrates the use of the aforementioned algorithms was imple-mented. The generated 3D object can easily be manipulated, e.g., rotated or translated, to render images from different viewpoints. This provides stable scenes of a minimal area that made it possible to understand the target space, and also made it easier for viewers t