EXEMPLAR BASED METRIC LEARNING FOR ROBUST VISUAL LOCALIZATION

This paper presents an exemplar based metric learning framework dedicated to robust visual localization in com-plex scenes, e.g. street images. The proposed framework learns off-line a specific (local) metric for each image of the database, so that the distance between a database image and a query image representing the same scene is smaller than the distance between the current image and other images of the database. To achieve this goal, we generate geometric and photometric transformations as proxies for query im-ages. From the generated constraints, the learning problem is cast as a convex optimization problem over the cone of positive semi-definite matrices, which is efficiently solved using a projected gradient descent scheme. Successful ex-periments, conducted using a freely available geo-referenced image database, reveal that the proposed method significantly improves results over the metric in the input space, while being as efficient at test time. In addition, we show that the model learns discriminating features for the localization task, and is able to gain invariance to meaningful transformations. Index Terms — content-based image retrieval, supervised metric learning, visual localization, place recognition 1. CONTEXT The problem tackled in this paper is visual localization at a street level [1] [2] [3]. Visual localization methods differ from each other by the type of features extracted online (2D only [2] and/or 3D [4]), the matching method (dedicated or not to large scale [5] [6]), and the a priori information used (geo-referenced database image [2], 3D model [3], 2D road-map [7], mobile cell phone information [6],...). Our typical scenario is the precise localization of a vehicle whose ap-proximate localization is known. Our problem is cast to an image retrieval (IR) problem using only 2D features extracted from acquired images and 2D geo-referenced database im-age (Fig. 1). Exploiting only image content is challenging because, even if query and database images depict the same scene, camera-view points, illumination and colorimetry are different, the scene itself may have changed over time an