Real-time Inserting Virtual Characters into Dynamic Video Scene

National audienceThe integration of moving virtual objects into real live-action scenes is an emerging topic. One difficult problem is to detect the presence of real moving obstacles, such as real people walking around, and to control the motion of virtual objects accordingly. This problem is however important to achieve geometrically coherent mixed reality content and to avoid irrealistic collisions between real and virtual objects. We propose a video-based solution. First, by analyzing the input video in real time, the motion states of the real targets are recovered into real 3D world. Secondly, under the constraint of the real dynamic targets, the virtual objects are controlled to move around and avoid the real targets in real time. Finally, to compute the visibility of each target, we extract each targets silhouette accurately which guarantees the computation of occlusion and location for the targets. Several examples demonstrate the efficacy of the proposed algorithm