Deformation Tracking in Depth and Color Video: An Analysis by Synthesis Approach

The tracking of deforming objects and the reconstruction of deformation in image sequences is one of the current research areas in computer vision. In contrast to rigid scenes, which can be analyzed and reconstructed very well, general deformations come with an infinite number of sub-movements and ways to parametrize them, which makes it very difficult to formulate discrete tracking goals. In contrast to the classic reconstructions based on color data alone, the combination of depth and color video provides tracking algorithms with a data foundation with less room for ambiguities, but also requires new algorithmic approaches to handle different entities and to exploit the available data. This thesis discusses an Analysis by Synthesis (AbS) scheme as an approach to the deformation tracking problem, a method that differs in many key aspects from common reconstruction schemes. It is demonstrated that AbS based deformation reconstruction can reconstruct complex deformations, deal with occlusions and self-occlusions, and can also be used for real-time tracking