Structure-aware computational imaging

While traditional imaging systems directly measure scene properties, computational imaging systems add computation to the measurement process, allowing such systems to extract non-trivially encoded scene features. This dissertation demonstrates that exploiting structure in this process allows to even recover information that is usually considered to be completely lost. Relying on temporally and spatially convolutional structure, we extract two novel image modalities that were essentially “invisible” before: a new temporal dimension of light propagation, and a new per-pixel radial velocity dimension, both obtained using consumer Time-of-Flight cameras. These two novel types of images represent first steps toward the inversion of light transport. Specifically, we demonstrate that Non-Line-of-Sight imaging and imaging in scattering media can be made feasible with additional temporal information. Furthermore, structure-aware imaging also represents a completely new approach to traditional color image processing. We show that classical hand-crafted image processing pipelines can be replaced by a single optimization problem exploiting image structure. This approach does not only outperform the state-of-the-art for classical image processing problems, but enables completely new color camera designs. In particular, we demonstrate camera designs with radically simplified optical systems, as well as novel sensor designs. The computation for all imaging problems from this dissertation relies on Bayesian inference using large-scale proximal optimization methods. We present a mathematical framework and a corresponding domain-specific language to automate the development of efficient, structure-aware solvers, allowing to immediately apply the insights gained in this dissertation to new imaging problems.Science, Faculty ofComputer Science, Department ofGraduat