Real-time Dense Simultaneous Localisation and Mapping over Large Scale Environments

The ability for a robot to create a map of an unknown environment and localise within that map is of critical importance in intelligent autonomous operation. This problem is referred to as Simultaneous Localisation and Mapping (or SLAM) and has been one of the major focusses of robotics research over the past 25 years. Although the initial focus was on 2D laser scan SLAM, more recently full 3D SLAM has become the dominant paradigm. The recent expansion in popularity of full, dense 3D SLAM is arguably a result of the release of the Microsoft Kinect commodity RGB-D sensor, which provides high quality depth sensing capabilities for a little over one hundred US dollars. Before the advent of the Kinect, 3D SLAM methods required either time of flight (TOF) sensors, 3D lidar scanners or stereo vision, which were typically either quite expensive or not suitable for fully mobile real-time operation if dense reconstruction was desired. Another recent technology which is often coupled with dense methods is General- Purpose computing on Graphics Processing Units (GPGPU) which exploits the massive parallelism available in GPU hardware to perform high speed and often real-time processing on entire images every frame. Being an affordable commodity technology, GPU-based programming is arguably another large enabler in recent dense SLAM research. Many visual SLAM systems and 3D reconstruction systems (both offline and online) have been published in recent times that rely purely on RGBD sensing capabilities because of the Kinect’s low price and accuracy; [43, 26, 113, 86]. However given the density of the data available, many existing systems have one or many limitations imposed by the challenges of processing such large amounts of information. These include a limitation in operating area, the inability to function in real-time over large scales, or not producing a globally consistent reconstruction of the explored environment or a map representation which is meaningful for robotic operations. In this thesis we address these issues through the development of a system which allows real-time globally consistent dense mapping over large scales, while providing a map representation which is useful for both autonomous robot navigation and higher level functionality such as object detection. The development of this system involves solving a number of critical issues including efficient real-time dense mapping over large scales, robust real-time camera pose estimation, a scalable means of correcting dense reconstructions for global consistency and representing the map in a format suitable for robotic operations. We address these issues respectively by 1) employing an efficient rolling cyclical buffer representation for mapping in the local frame; 2) estimating a dense photometric camera pose constraint in conjunction with a dense geometric constraint and jointly optimising for a camera pose estimate; 3) optimising the dense map by means of a non-rigid space deformation parameterised by a loop closure constraint; and, 4) intelligently simplifying the dense map reconstruction to a planar representation. As part of this the system is implemented as a set of hierarchical multithreaded components which are capable of operating in real-time. The architecture facilitates the creation and integration of new modules with minimal impact on the performance of the overall system. This yields an adaptable and easily extendable system which is easily combined with other software systems designed for related operations. We provide a comprehensive quantitative and qualitative evaluation of all aspects of the system’s performance, demonstrating real-time dense SLAM over large scales. Our evaluation includes comparisons to other approaches on standard benchmarks in terms of computational performance, trajectory estimation and surface reconstruction quality