Robot Guidance by Human Pointing Gestures

In the search for intuitive man-machine interfaces the visual recognition of human gestures would provide a powerful means to guide robot movements. In our paper we report on the development of the modular neural system "See-Eagle" for the visual guidance of robot pick-and-place actions. Several neural networks are integrated to a single system that visually recognizes human hand pointing gestures from stereo pairs of color video images. The accuracy of the pointing recognition system alone allows to estimate the pointing target to an accuracy of 2 cm in a workspace area of 50 \Theta 50 cm. The output of the hand recognition stage is further processed by a set of color-sensitive neural networks to determine the cartesian location of the target object that is referenced by the pointing gesture. Finally, this information is used to guide a robot to pick the target object and place it at another location that can be specified by a second pointing gesture. The accuracy of the combined syst..