Multimodal object perception for robotics

The ability to recognize and manipulate unknown objects is crucial for any robot to successfully work in natural environments. Object recognition and categorization is a very challenging problem, as three-dimensional objects often give rise to ambiguous, two-dimensional views. This dissertation investigates how multisensory integration methods can be employed to tackle this issue in robotic applications. First, the presented approaches are motivated by discussing aspects of biological and computational object perception. To increase the understanding of the technical context, concepts regarding software development on modern robotic systems will be outlined. The document is separated into two parts, representing two strategies for multimodal object recognition utilizing state-of-the-art robotic hardware. One part focuses on the benefits of employing range sensing technology for the categorization of everyday objects. We present a novel object dataset that served as a testbed to study classification performance combining 2D and 3D cues. The second part incorporates object manipulation into the recognition process. A perception driven object exploration method, implemented on the humanoid robot iCub, will be presented. In this setup, the robot turns and moves an object in its hand in order to seek out informative views, thereby optimizing the exploration sequence. It will also be shown that, instead of relying on purely visual information, taking motor actions that link object views into account allows the robot to resolve a significant amount of ambiguity