From tele-operated robots to social robots with autonomous behaviors

This paper presents a perception/interface device for Telepresence Mobile Robots using a Kinect sensor. Firstly, using the Kinect RGB camera (Webcam) and image processing techniques, it is possible to detect a human face, allowing the robot to track the face, getting closer of a person, moving forward and rotating to get a better pose (position and orientation) to interact with him/her. Then it is possible to recognize hand gestures using the Kinect 3D sensor (Depth camera). The proposed gesture recognition method tracks the hands positions and movements, when moving it forward towards the robot, and then recognizing a set of predefined gestures/commands. Finally, the 3D perception provided by the Kinect also allows us to detect obstacles, avoiding collisions and safely moving the mobile robot base, also allowing to search for someone in the environment. Practical experiments are presented demonstrating the obtained results: (i) in the searching and tracking of human faces; (ii) in the robot positioning related to the user we want to interact with; and also (iii) in the human gesture recognition.CAPES/D