Fast Semantic Segmentation of RGB-D Scenes with GPU-Accelerated Deep Neural Networks

Abstract. In semantic scene segmentation, every pixel of an image is assigned a category label. This task can be made easier by incorporat-ing depth information, which structured light sensors provide. Depth, however, has very different properties from RGB image channels. In this paper, we present a novel method to provide depth information to convo-lutional neural networks. For this purpose, we apply a simplified version of the histogram of oriented depth (HOD) descriptor to the depth chan-nel. We evaluate the network on the challenging NYU Depth V2 dataset and show that with our method, we can reach competitive performance at a high frame rate