Foreground Detection with a Moving RGBD Camera

Abstract. A method for foreground detection in data acquired by a moving RGBD camera is proposed. The background scene is initially in a reference model. An initial estimation of camera motion is provided by a conventional point cloud registration approach of matched keypoints between the captured scene and the reference model. This initial solution is then refined based on a top-down, model based approach that evaluates candidate camera poses in a Particle Swarm Optimization framework. To evaluate a candidate pose, the method renders color and depth images of the model according to this pose and computes a dissimilarity score of the rendered images to the currently captured ones. This score is based on the direct comparison of color, depth, and surface geometry between the acquired and rendered images, while allowing for outliers due to the potential occurrence of foreground objects, or newly imaged surfaces. Extended quantitative and qualitative experimental results confirm that the proposed method produces significantly more accurate foreground segmentation maps compared to the conventional, baseline feature-based approach.