Interactive image-based 3D reconstruction techniques for application scenarios at different scales

The image-based 3D reconstruction of real-world objects or scenes has attracted considerable attention in recent years, and several approaches capable of generating high quality results have been proposed. However, since most of these systems aim at being fully automatic without the ability to take user input into account, the key problem of existing image-based 3D reconstruction approaches is the insufficient handling of errors and failure cases. As a consequence, in practice even low error rates of an automatic reconstruction algorithm result in considerable and tedious manual post-processing efforts. To address this issue, in this thesis, we are developing image-based 3D reconstruction techniques with a focus on interactive systems. For different reconstruction scenarios, we present effective, problem-specific user-interfaces and demonstrate their practical applicability at various examples. Clearly, different 3D reconstruction scenarios have very different requirements with respect to the image capturing process, the representation of the model's geometry, and the actual reconstruction techniques. In order to meet these requirements, we distinguish three different scenarios by the actual size of the object or scene to be reconstructed, and present suitable geometry representations and reconstruction algorithms. Specifically, we discuss approaches for the reconstruction of individual objects, indoor environments, and complete cities, respectively. In the first part of the thesis, we present two novel techniques for the reconstruction of individual objects. The first is a surface growing approach that employs a greedy expansion of already recovered surface information into unknown regions, with particularly low requirements on the input data. The second then extends the idea of 3D reconstruction by surface expansion to an interactive system. Here, the main distinguishing feature is a user interface which is based on a small set of intuitive 2D painting metaphors to guide and control the reconstruction process. We demonstrate at various examples that the interactive approach not only considerably reduces the overall processing time, but is capable of strongly improving the quality of the resulting models. The second part of the thesis focuses on the reconstruction of indoor environments. To be able to cope with complex surface shapes and varying, often completely textureless surface materials, we propose a novel active laser-based depth map reconstruction approach. In contrast to standard laser-based reconstruction techniques that require the relation between camera and laser to be precisely calibrated, our system is based on a hand-held laser rig. By recovering the rig's pose for each frame of the input image sequence, we enable a flexible reconstruction approach. Finally, in the third part we consider the reconstruction of entire city regions based on oblique aerial images. Since standard extrinsic calibration approaches are not applicable to this image type, we first discuss an approach to precisely register a set of oblique aerial images with a cadastral map. Based on this input data, we then present two different interactive city modeling approaches with a major focus on compact geometry representations and simple, yet effective user interfaces