3-D Object Segmentation from Motion Cues

Thesis (Ph.D.)--University of Washington, 2014In this work we tackle the problem of segmenting rigid objects using various types of motion cues. The setting is that of personal robotics, in which robots generally have cameras and arms, and are capable of manipulating objects at close range. Object segmentation is the problem of determining the boundaries, in a 2-D view or full 3-D, of objects present in a scene. In this work we make use of motion caused both by the robot and by other agents to perform segmentation. Similarly, we can use motion for segmentation whether the actual movement was observed by a robot or not. Our motion estimation algorithms are based on change detection, point feature matching, and scene flow. We build on recent static online SLAM techniques to perform online segmentation and 3-D modeling of rigid objects. Performing segmentation online allows us to also do active segmentation, in which a robot moves its camera and/or manipulates objects in the scene in order to improve its segmentation and/or geometric object models. We perform object discovery on the partial object models we obtain from any of these segmentation methods. We design an objective function using shape and color information to be minimized to align partial object models, and we can use information from this alignment process for clustering of object views into instances. This gives us further information on the movement of individual objects over time. All of this work is performed with RGB-D cameras, which capture both color and depth images. Our methods are designed to work with RGB-D image data and to handle the behavior of RGB-D sensors with respect to missing data. We present an accurate probabilistic noise model for RGB-D cameras and most of our work builds on it