Wide baseline stereo matching based on local, affinely invariant regions

‘Invariant regions ’ are image patches that automatically deform with chang-ing viewpoint as to keep on covering identical physical parts of a scene. Such regions are then described by a set of invariant features, which makes it rela-tively easy to match them between views and under changing illumination. In previous work, we have presented invariant regions that are based on a com-bination of corners and edges. The application discussed then was image database retrieval. Here, an alternative method for extracting (affinely) invariant regions is given, that does not depend on the presence of edges or corners in the image but is purely intensity-based. Also, we demonstrate the use of such regions for another application, which is wide baseline stereo matching. As a matter of fact, the goal is to build an opportunistic system that exploits several types of invariant regions as it sees fit. This yields more correspondences and a system that can deal with a wider range of images. To increase the robustness of the system even further, two semi-local con-straints on combinations of region correspondences are derived (one geomet-ric, the other photometric). They allow to test the consistency of correspon-dences and hence to reject falsely matched regions.