Digitizing human performance with robust range image registration

2017-06-26The rekindling of interest in Augmented Reality and Virtual Reality has birthed a need for digitizing objects with full geometry and texture, especially human body and human performance. Recent advance of commodity depth sensors (e.g., Kinect One and Occipital IO) has quickly gained popularity in both research community and industry since they are cost efficient, accessible while providing decent depth measurements at an interactive frame rate. Registration, which aims to transform all range images into the same coordinate system, plays a key role in the 3D digitization process, as a small error in registration will lead to a large degradation of final reconstruction quality. Registration is well addressed by existing techniques when the two views exhibit significant overlap and when a rough registration between them is given. Different methods have been proposed to relax these conditions, no satisfactory solution exists, however, to consistently address the combined problem. While all previous methods perform registration directly on the generated point clouds, we propose here a novel framework to explore the visibility information of the underlying range images. This allows us to efficiently handle the problem of insufficient overlap. In this dissertation, we adopt the visibility information to solve the problems of rigid registration, non-rigid registration and global registration respectively. This enables a 3D modeling system of rigid as well as non-rigid objects from as few as 4 range images, while other traditional 3D modeling methods normally require a few hundred range images. Experimental results on both synthetic and real data demonstrate the effectiveness and robustness of our registration methods, as well as our human performance digitization systems