Panoramic Mosaics by Manifold Projection

As the field of view of a picture is much smaller than our own visual field of view, it is common to paste together sev-eral pictures to create a panoramic mosaic having a larger field of view. Images with a wider field of view can be gen-erated by using fish-eye lens, or panoramic mosaics can be created by special devices which rotate around the camera’s optical center (Quicktime VR, Surround Video), or by align-ing, and pasting, frames in a video sequence to a single reference frame. Existing mosaicing methods have strong limitations on imaging conditions, and distortions are com-mon. Manifold projection enables the creation of panoramic mosaics from video sequences under more general condi-tions, and in particular the unrestricted motion of a hand-held camera. The panoramic mosaic is a projection of the scene into a virtual manifold whose structure depends on the camera’s motion. This manifold is more general than the customary projections onto a single image plane or onto a cylinder. In addition to being more general than traditional mo-saics, manifold projection is also computationally efficient, as the only image deformations used are image-plane trans-lations and rotations. Real-time, software only, implemen-tation on a Pentium-PC, proves the superior quality and speed of this approach.