3D Object Modeling and Recognition in Photographs and Video

This thesis introduces a novel representation for three-dimensional (3D) objects in terms of local affine-invariant descriptors of their appearance and the spatial relationships between the corresponding affine regions. Geometric constraints associated with different views of the same surface patches are combined with a normalized representation of their appearance to guide matching and reconstruction, allowing the acquisition of true 3D models from multiple unregistered images, as well as their recognition in photographs and image sequences. The proposed approach is applied to two domains: 1) Photographs -- Models of rigid objects are constructed from photos and recognized in highly cluttered shots taken from arbitrary viewpoints. 2) Video -- Dynamic scenes containing multiple moving objects observed by a moving camera are segmented into rigid components, and the 3D models constructed from these components are matched across different image sequences, with application to shot matching