Virtual viewpoint replay for a soccer match by view interpolation from multiple cameras

Abstract—This paper presents a novel method for virtual view synthesis that allows viewers to virtually fly through real soccer scenes, which are captured by multiple cameras in a stadium. The proposed method generates images of arbitrary viewpoints by view interpolation of real camera images near the chosen viewpoints. In this method, cameras do not need to be strongly calibrated since projective geometry between cameras is employed for the interpo-lation. For avoiding the complex and unreliable process of 3-D re-covery, object scenes are segmented into several regions according to the geometric property of the scene. Dense correspondence be-tween real views, which is necessary for intermediate view genera-tion, is automatically obtained by applying projective geometry to each region. By superimposing intermediate images for all regions, virtual views for the entire soccer scene are generated. The efforts for camera calibration are reduced and correspondence matching requires no manual operation; hence, the proposed method can be easily applied to dynamic events in a large space. An application for fly-through observations of soccer match replays is introduced along with the algorithm of view synthesis and experimental re-sults. This is a new approach for providing arbitrary views of an entire dynamic event. Index Terms—Dynamic event, multiple cameras, projective ge-ometry, soccer match, view interpolation, virtual view synthesis. I