Feature Densities are Required for Computing Feature Correspondences

The feature correspondence problem is a classic hurdle in visual object-recognition concerned with determining the correct mapping between the features measured from the image and the features ex-pected by the model. In this paper we show that determining good correspondences requires information about the joint probability density over the image features. We propose "likelihood based correspondence matching " as a general principle for selecting op-timal correspondences. The approach is applicable to non-rigid models, allows nonlinear perspective transformations, and can op-timally deal with occlusions and missing features. Experiments with rigid and non-rigid 3D hand gesture recognition support the theory. The likelihood based techniques show almost no decrease in classification performance when compared to performance with perfect correspondence knowledge.