Structure and motion estimation from rolling shutter video

The majority of consumer quality cameras sold today have CMOS sensors with rolling shutters. In a rolling-shutter camera, images are read out row by row, and thus each row is exposed during a different time interval. A rolling-shutter exposure causes geometric image distortions when either the camera or the scene is moving, and this causes state-of-the-art structure and motion algorithms to fail. We demonstrate a novel method for solving the struc-ture and motion problem for rolling-shutter video. The method relies on exploiting the continuity of the camera mo-tion, both between frames, and across a frame. We demon-strate the effectiveness of our method by controlled experi-ments on real video sequences. We show, both visually and quantitatively, that our method outperforms standard struc-ture and motion, and is more accurate and efficient than a two-step approach, doing image rectification and structure and motion. 1