Multi-session Visual Simultaneous Localisation and Mapping

One of the principal aims of robotics is to develop robots that are capable of long term autonomy in unstructured and unknown environments. Such autonomy will only be achieved through algorithms that permit robots to perceive, interpret, and interact with the world they inhabit. The foundation to such algorithms is the ability to build and maintain a map of the environment and to estimate the robot’s location relative to that map. This problem is referred to as Simultaneous Localisation and Mapping (or SLAM). Over the past 25 years considerable progress has been made on the SLAM problem with a large number of solutions being reported in the literature. Although the majority of earlier systems depended on active ranging and proprioceptive sensors, more recently multiple approaches have been reported that rely purely on visual sensors. Visual sensors provide much richer measurements of the environment and bring with them a wealth of techniques from the field of computer vision in areas such as feature detection, tracking, and image matching. However, despite substantial recent progress in visual SLAM [103], many issues remain to be solved before a robust, general visual mapping and navigation solution can be widely deployed. Central among these issues is persistence - the capability for a robot to operate robustly for long periods of time. As a robot makes repeated transits through previously visited areas, it cannot simply treat each mission as a completely new experiment, not making use of previously built maps. However, nor can the robot treat its complete lifetime experience as “one big mission”, with all data considered as a single pose graph and processed in a single batch optimisation. In this thesis this problem is addressed through the development of a framework that achieves a balance between these two extremes, enabling the robot to leverage off the results of previous missions, while still adding in new areas as they are uncovered thereby improving its map over time. The contribution of this thesis is the development of system for performing real-time multi-session visual mapping in large-scale environments. Multi-session mapping considers the above problem i.e. combining the results of multiple simultaneous localisation and mapping (SLAM) missions performed repeatedly over time in the same environment. The goal is to robustly combine multiple maps in a common metrical coordinate system, with consistent estimates of uncertainty. Our work employs incremental smoothing and mapping (iSAM) as the underlying SLAM state estimator and uses an improved appearance-based method for detecting loop closures within single mapping sessions and across multiple sessions. A critical issue is how to pose the state estimation problem for combining the results of multiple mapping missions efficiently and robustly. We solve this problem by keeping each mission in its own relative frame of reference and employ spatial separator variables, called anchor nodes, to link together these multiple relative pose graphs. The system architecture consists of a separate front-end for computing visual odometry and windowed bundle adjustment on individual sessions, in conjunction with a back-end for performing the place recognition and multi-session mapping. We provide a comprehensive quantitative analysis of the system’s performance, demonstrating real-time multi-session visual mapping. The experimental datasets were captured using wheeled and handheld cameras and include indoor, outdoor, and mixed sequences captured over large-scale environments