Predictive sensing for field robotics

The global autonomous robot market is expected to be worth more than eleven billion US dollars by 2024, with a need - across industries - for autonomous robots able to operate safely in complex and dynamic real-world environments. In this thesis, we argue that this will require developing robots able to develop situational awareness by constructing a higher level understanding of the world beyond the instantaneous and limited data provided by their sensors. We suggest this can be achieved by enabling robots to learn directly from their environment to predict the future evolution of the world and performance of their systems. Robust mobile autonomy is developed around the three pillars of navigation, perception, and planning. For an agent to effectively plan a route through its environment, it needs to know where it is in its environment and what is going on around it. To do so, it is equipped with localisation and perception systems which can interpret incoming data from onboard sensors. Localisation systems typically provide instantaneous information with regards to whether an agent is localised or lost. Perception systems typically tracks objects in the environment using multi-stage pipelines which operate independently, require much hand-engineering, and show little robustness to natural occlusions. These low-level interpretations of incoming data often contribute to poor situational awareness and make it difficult to plan ahead with far-sightedness. In this thesis we address both shortcomings and propose data-driven approaches to increase situational awareness for a localisation and perception system operating in the field. In the area of navigation, we propose a novel framework for predicting ahead of time how well a robot will be able to localise in a given environment given an appearance model of the scene. In the area of perception, we extend an end-to-end deep-learning framework for predicting near-future scene occupancy beyond natural occlusions, to operate in the real world, from a moving platform, and taking into consideration the scene context. In doing so, we contribute to developing greater situational awareness for robotic systems operating in real world environments. We argue that robotic systems can make greater sense of what their are perceiving by moving away from instant sensing to predictive sensing.