Multiview 3D Sensing and Analysis for High Quality Point Cloud Capturing and Model Generation.

PhD Theses.Multi-view 3D reconstruction techniques enable digital reconstruction of 3D objects from the real world by fusing di erent viewpoints of the same object into a single 3D representation. However, this task is not trivial as the RGB-D sensors produce vast amounts of inaccurate depth data that needs to be processed in real-time. The real-time aspect of this work is vital as the reconstruction system connects multiple sensors into a single computer. Moreover, the quality of the 3D reconstruction is highly dependent on the extrinsic parameters between sensors. Addressing these issues placed on multi-view real-time point cloud reconstruction, this thesis proposes the use of a low-cost 3D studio environment which enables photorealistic reconstruction of human avatars and the environment. The Multi-view Reconstruction System (MRS) presented in this thesis exploited the connection of three RGB-D cameras into single machine that is able to provide real-time 3D reconstruction of human avatars. It is further divided into the following four steps: data acquisition, calibration, point cloud enhancement and visualisation. A key challenge when displaying and processing sensed real-time 3D data is the e ciency of post-processing algorithms in order to display 3D content at high quality. This process is by no means trivial and requires well designed algorithms for fast processing and quick rendering of point cloud data. Therefore, our system uses a multi-threaded pipeline in order to acquire depth data separately for each device, then fuse them into a single point cloud representation. Moreover, we investigated calibration procedures of multiple RGB-D sensors through standard calibration algorithms. Therefore, this thesis introduces a unique calibration approach using a double-sided checkerboard pattern that is extended with 3D depth sensor data. To validate the calibration results, we compared our results with other calibration methods. Finally, a postprocessing step has been introduced, that utilises depth sensor noise measurements to estimate depth pixel liability of reconstructed 3D points in order to increase the accuracy of the acquired depth map. The performance of the proposed approach for the reconstruction has been evaluated by using two setups with three sensors, namely Kinect V2 and RealSense D435