Free Viewpoint Video Synthesis based on Visual Hull Reconstruction from Hand-Held Multiple Cameras

Abstract—This paper proposes a novel method for synthesizing free viewpoint video which is captured by hand-held multiple cameras. Cameras in our system are all uncalibrated. Neither intrinsic nor extrinsic parameters are known. Projective Grid Space (PGS) which is 3D space defined by epipolar geometry of two basis cameras is used for dynamic cameras calibration. Geometrical relations among cameras in PGS are obtained from 2D-2D corresponding points between views. We utilize Scale Invariant Feature Transform (SIFT) for finding corresponding points in natural scene for registering cameras to PGS. Moving object is segmented via graph cut optimization. Finally, free viewpoint video is synthesized based on the reconstructed visual hull. In the experimental results, free viewpoint video which is captured by hand-held cameras is successfully synthesized using the proposed method. I