An Augmented Reality System for Urban Environments using a Planar Building Facade Model

In recent years there has been a widespread adoption of smartphone technology. According to research by Google [Pha12] the U.S., New Zealand, Denmark, Ireland, Netherlands, Spain and Switzerland have a smartphone penetration greater than 40 percent. These phones are frequently equipped with with reasonable processing capabilities, internet access, high resolution cameras, and positioning sensors. This widespread adoption coupled with the hardware capability\u27s of current generation smartphones present a great opportunity to develop mobile augmented reality (AR) applications for visualising geospatial information. While smartphones are able to provide geolocalisation through GPS, the lack of accuracy does not allow for precise and robust alignment of the augmented data with the camera image. Computer vision techniques can be used to gain accurate alignment. However, these methods often require the creation a comprehensive 3D model of a scene. This can be intensive to compute and the storage of this data can become a problem when the system is required to work over the scale of a citywide area. This thesis presents a mobile AR navigation system that uses a planar based representation in order to augment 3D data into an urban scene. By performing planar extraction the system can use the facades of building as reference for accurately augmenting content. This is turn removes the need for building a comprehensive 3D model for the scene. The system uses a client server architecture where a user takes an image within an urban setting with their smartphone which is forwarded to the server for processing. This processing includes extraction of the planar facades in the scene, camera pose estimation and, matching of the facades to a topological map of the environment. Once matched each of the the associated camera pose parameters in conjunction with relevant AR content can then be sent back to the mobile device. These parameters allow the system to integrate the augmented content into the view which can highlight and visualize geographically or contextually meaningful information about the scene. Results presented demonstrate that the system provides an accurate and robust approach to AR in urban environments without associated complexity of creating metric 3D reconstructions of the environment