On the Combination of {KLT} Tracking and {SIFT} Matching

Finding correspondences is a crucial aspect in many fields of computer vision and computer graphics such as structure from motion, camera motion estimation and 3D reconstruction. Current feature point detection and motion tracking algorithms provide accurate correspondences for a sequence of images. However, if the corresponding 3D point of some feature track is occluded, leaves the image or is rejected for some other reason, the feature track is dropped. If the point reappears in some later image, a new track is started without knowing of the existence of the old track, thus losing important information about the scene and the motion of the point. There exists no single algorithm that allows to track feature points in a short range as well as long range.\\ We propose an algorithm that takes advantage of both, optic flow based feature point tracker and descriptor based long range matching. While the feature point tracker provides accurate feature tracks, we use descriptor based matching to combine new tracks with already existing tracks, thereby reducing the number of feature tracks covering the whole scene while increasing the number of feature points per track. Feature tracks are represented by a minimal amount of feature descriptors that describe the feature track best