Feature guided pixel matching and segmentation in motion image sequences

The problem of feature correspondences and trajectory finding for a long image sequence has received considerable attention. In this research, a coarse-to-fine algorithm is developed to obtain pixel trajectories through the sequence and to segment them into subsets corresponding to objects moving distinctly. First, a coarse-scale point-feature detector is used to detect point features that are then used to form a 3D dot pattern in the spatio-temporal region. The trajectories are extracted as 3D curves formed by the points using perceptual grouping. The set of feature points in each frame is divided into subsets corresponding to objects moving with different motion, using a measure of motion similarity between feature points.Next, increasingly dense correspondences are obtained iteratively from the initial matches for sparse point features. A Delaunay triangulation of the matched features in each frame is computed. Additional point features having higher densities are detected. The motions of these denser features are predicted based on the known motions of nearby, coarser-level features. The coarser-level features near a detected fine-level feature may belong to one or more objects with different motions. All different motions are considered, and candidate matches are computed using gray-level correlation. The relaxation algorithm is used to select the best candidate for each feature point. These finer-level correspondences can again be segmented into objects moving distinctly in the same way as was done at the coarser level. This is followed by a reiteration of the process of taking more feature points, predicting their motions, computing candidate matches, selecting the best match, and segmenting these finer-level feature points into objects moving distinctly. The coarse-to-fine level iteration is repeated until the feature detector no longer provides useful new features.Once the finest-level features are found and matched, the matching of all pixels is done using intensity correlation. Again, a pair of frames is considered for this purpose. A Delaunay triangulation is computed for the matched features at the finest level in a frame. The three vertices of a Delaunay triangle may belong to one, two, or three objects moving with different motions. All of the motions are considered for computing candidate matches for each pixel in a triangle. The relaxation algorithm is used to obtain the best match in a way similar to that used for finer-level matching. Once the pixel-level matches are available between two frames, an attempt is made to obtain the finest boundaries of the moving objects. The results of feature-point matching at the finest level are used to extend matches. Pixel-level matches are computed from the results of finest-level point-feature correspondences between a pair of frames. The batches of overlapping frames are formed and processed as described to obtain the results for an entire sequence.U of I OnlyETDs are only available to UIUC Users without author permissio