Outdoors Augmented Reality on Mobile Phone using Loxel-Based Visual Feature Organization

We have built an outdoors augmented reality system for mobile phones that matches camera-phone images against a large database of location-tagged images using a robust image retrieval algorithm. We avoid network latency by implementing the algorithm on the phone and deliver excellent performance by adapting a state-of-the-art image retrieval algorithm based on robust local descriptors. Matching is performed against a database of highly relevant fea-tures, which is continuously updated to reflect changes in the en-vironment. We achieve fast updates and scalability by pruning of irrelevant features based on proximity to the user. By compress-ing and incrementally updating the features stored on the phone we make the system amenable to low-bandwidth wireless connections. We demonstrate system robustness on a dataset of location-tagged images and show a smart-phone implementation that achieves a high image matching rate while operating in near real-time