Indoor camera relocalization under changing conditions using attention-based neural networks

To perform tasks autonomously a robot oftentimes needs to be able to localize itself. One specific form of localization is relocalization which describes the process of estimating the position only based on the current observation and some kind of map of the environment. This is necessary if the robot has lost track of its current position or is placed in a new scene. In this thesis we concentrate on visual relocalization. Hereby, the observation of the robot is given in the form of an image that we call query image and also the scene is represented as a collection of images, called reference images, with their corresponding absolute pose. Based on the collection of reference images, the absolute pose of the query image has to be estimated. A big variety of approaches has been proposed to solve visual relocalization, but most of them are scene-specific. This means they need a detailed 3D model of the environment or in the case of learning-based approaches need to be retrained for each new scene. For big environments or emergency operations in unseen scenes this can be too expensive or simply not possible. Therefore, we want to concentrate on scene-agnostic relocalization in this thesis. Deep Learning has been successfully applied to many different fields and can also be used for visual relocalization. Unfortunately, the generalization capability of such methods to unseen scenes is still outperformed by non-learning based approaches. In this thesis we show that one of the main reasons for the bad accuracy on unseen scenes is the too-small pose regression part of existing pose estimation networks. By extending this regression part, our novel network architecture called \textit{ExReNet} successfully generalizes from synthetic training data to real-world data. To be able to compare our model to existing approaches, we evaluate our model on the 7-Scenes dataset. Thereby, we outperform all previous deep learning-based approaches that were trained on unrelated data and most of the approaches that were specifically trained on the 7-Scenes dataset. Furthermore, we show that the pose estimation can be further improved by taking translation scale and uncertainty into account. Hereby, we propose a novel approach to model uncertainty via a variational autoencoder. Next to 7-Scenes, we also use a Replica-based dataset to evaluate our approach in a more realistic scenario where only a few reference images are available. We show that under such much more realistic conditions our learning-based approach can outperform the classical non-learning-based methods and by that also the existing learning-based methods