Realtime motion segmentation based multibody visual slam

In this paper, we present a practical vision based Simultane-ous Localization and Mapping (SLAM) system for a highly dynamic environment. We adopt a multibody Structure from Motion (SfM) approach, which is the generalization of classical SfM to dynamic scenes with multiple rigidly mov-ing objects. The proposed framework of multibody visual SLAM allows choosing between full 3D reconstruction or simply tracking of the moving objects, which adds flexibil-ity to the system, for scenes containing non-rigid objects or objects having insufficient features for reconstruction. The solution demands a motion segmentation framework that can segment feature points belonging to different motions and maintain the segmentation with time. We propose a re-altime incremental motion segmentation algorithm for this purpose. The motion segmentation is robust and is capa-ble of segmenting difficult degenerate motions, where the moving objects is followed by a moving camera in the same direction. This robustness is attributed to the use of ef-ficient geometric constraints and a probability framework which propagates the uncertainty in the system. The mo-tion segmentation module is tightly coupled with feature tracking and visual SLAM, by exploring various feed-backs in between these modules. The integrated system can si-multaneously perform realtime visual SLAM and tracking of multiple moving objects using only a single monocular camera. 1