Learning invariant representations and applications to face verification

One approach to computer object recognition and modeling the brain’s ventral stream involves unsupervised learning of representations that are invariant to com-mon transformations. However, applications of these ideas have usually been lim-ited to 2D affine transformations, e.g., translation and scaling, since they are eas-iest to solve via convolution. In accord with a recent theory of transformation-invariance [1], we propose a model that, while capturing other common con-volutional networks as special cases, can also be used with arbitrary identity-preserving transformations. The model’s wiring can be learned from videos of transforming objects—or any other grouping of images into sets by their depicted object. Through a series of successively more complex empirical tests, we study the invariance/discriminability properties of this model with respect to different transformations. First, we empirically confirm theoretical predictions (from [1]) for the case of 2D affine transformations. Next, we apply the model to non-affine transformations; as expected, it performs well on face verification tasks requiring invariance to the relatively smooth transformations of 3D rotation-in-depth and changes in illumination direction. Surprisingly, it can also tolerate clutter “trans-formations ” which map an image of a face on one background to an image of the same face on a different background. Motivated by these empirical findings, we tested the same model on face verification benchmark tasks from the computer vision literature: Labeled Faces in the Wild, PubFig [2, 3, 4] and a new dataset we gathered—achieving strong performance in these highly unconstrained cases as well.