Deep Structured Models for Spatial Intelligence

The future robots and embodied devices will seamlessly interact with the physical world, help us in our everyday lives and become an integral part of society. To make this dream come true, we need to bring spatial intelligence to our embodied devices: the ability to understand, reason, and create rich knowledge about the spatial relations among objects and space. While computer vision has significantly enhanced capabilities in 2D perception, we are still missing key ingredients to achieve the desired goal — holistic spatial modeling of our 3D world. In this thesis, we present a collection of work towards this goal. The core idea is to build expressive representations of the world and exploit rich structure in the problems. Our research efforts span both creating new structured models as well as developing novel solutions to various applications. The effectiveness of our integrated approach has been demonstrated at the full spectrum of spatial intelligence tasks, including localization, perception, and mapping. Finally, I will give a brief personal outlook on open research topics towards achieving spatial intelligence.Ph.D