A framework for collaborative real-time 3D teleimmersion in a geographically distributed environment

In this paper, we present a framework for immersive 3D video conferencing and geographically distributed collab-oration. Our multi-camera system performs a full-body 3D reconstruction of users in real time and renders their image in a virtual space allowing remote interaction be-tween users and the virtual environment. The paper fea-tures an overview of the technology and algorithms used for calibration, capturing, and reconstruction. We introduce stereo mapping using adaptive triangulation which allows for fast (under 25 ms) and robust real-time 3D reconstruc-tion. The chosen representation of the data provides high compression ratios for transfer to a remote site. The algo-rithm produces partial 3D meshes, instead of dense point clouds, which are combined on the renderer to create a uni-fied model of the user. We have successfully demonstrated the use of our system in various applications such as remote dancing and immersive Tai Chi learning. 1