An integrated robotic system for spatial understanding and situated interaction in indoor environments

A major challenge in robotics and artificial intelligence lies in creating robots that are to cooperate with people in human-populated environments, e.g. for domestic as-sistance or elderly care. Such robots need skills that al-low them to interact with the world and the humans liv-ing and working therein. In this paper we investigate the question of spatial understanding of human-made envi-ronments. The functionalities of our system comprise perception of the world, natural language, learning, and reasoning. For this purpose we integrate state-of-the-art components from different disciplines in AI, robotics and cognitive systems into a mobile robot system. The work focuses on the description of the principles we used for the integration, including cross-modal integra-tion, ontology-based mediation, and multiple levels of abstraction of perception. Finally, we present experi-ments with the integrated “CoSy Explorer”1 system and list some of the major lessons that were learned from its design, implementation, and evaluation