Gesture Recognition for Human-Robot Interaction for Service Robots

Robots are quickly becoming an intrinsic part of our daily lives and it is becoming important to provide the users a simple and intuitive way to interact with them. In this thesis, we present a multimodal Human-Robot interface for an existing service robot, mostly addressed to people with reduced mobility on the shopping process in dynamic and crowded environments ( eg. supermarkets). This interface was created in order to recognize the "Start", "Stop" and "Pause" commands.The proposed Human-Robot Interface includes two types of interaction: verbal and non-verbal. Regarding verbal interaction, four state of the art implementations (Google Speech Recognition, Houndify, Microsoft Bing Voice Recognition, CMUsphinx) were tested and compared. The Houndify proved to be the more suitable for our project.Relatively to the non-verbal interaction, a novel method for hand gesture recognition based on depth information was implemented and tested. The software was developed to be used by a robot equipped with a RGB-D camera. This camera captures images in real time where the robot user's position is obtained. Taking as input the information already processed by the robot, the arm/hand is obtained by a depth based segmentation approach. A principal component analysis is then computed to each object where its center of mass and eigen vectors are calculated in order to extract the hand's tip and orientation. A Kalman filter is then applied for tracking the hand and get its position through time. Given this information and based on finite state machines that were implemented to describe gestures (start, stop, pause) we perform gesture recognition