Aligning 3D models to RGB-D images of cluttered scenes

eecs.berkeley.edu uniandes.edu.co microsoft.com eecs.berkeley.edu The goal of this work is to represent objects in an RGB-D scene with corresponding 3D models from a library. We ap-proach this problem by first detecting and segmenting object instances in the scene and then using a convolutional neural network (CNN) to predict the pose of the object. This CNN is trained using pixel surface normals in images containing renderings of synthetic objects. When tested on real data, our method outperforms alternative algorithms trained on real data. We then use this coarse pose estimate along with the inferred pixel support to align a small number of pro-totypical models to the data, and place into the scene the model that fits best. We observe a 48 % relative improve-ment in performance at the task of 3D detection over the current state-of-the-art [34], while being an order of mag-nitude faster. 1