Putting local features on a manifold

Local features have proven very useful for recognition. Manifold learning has proven to be a very powerful tool in data analysis. However, manifold learning application for images are mainly based on holistic vectorized representa-tions of images. The challenging question that we address in this paper is how can we learn image manifolds from a punch of local features in a smooth way that captures the feature similarity and spatial arrangement variability be-tween images. We introduce a novel framework for learn-ing a manifold representation from collections of local fea-tures in images. We first show how we can learn a feature embedding representation that preserves both the local ap-pearance similarity as well as the spatial structure of the features. We also show how we can embed features from a new image by introducing a solution for the out-of-sample that is suitable for this context. By solving these two prob-lems and defining a proper distance measure in the feature embedding space, we can reach an image manifold embed-ding space. 1