Learning to comprehend deictic gestures in robots and human infants

Abstract — In this paper, I discuss how visual information about deictic gestures influences learning that enables these gestures to be comprehended. It has been suggested that the ability of human infants to comprehend deictic gestures depends on the physical appearance of gestures, the move-ment of gestures, and the distance between gestures and the indicated targets. To understand the mechanisms involved, I apply a model that enables a robot to recognize the static orientation of a gesture as an edge image and movement as optical flow. Experiments using a robot reveal that (1) learning to comprehend reaching gestures with all fingers extended is more accelerated than learning to comprehend pointing with the index finger, and that (2) downward tapping movement facilitates learning more than pointing movement along the direction of the gesture. These results suggest that (1) the quantitative difference in the edge features of reaching and pointing that correspond to the directions of gestures influ-ences learning speed, and that (2) the optical flow of tapping movement that offers qualitatively different information from that provided by the edge image makes learning easier than the optical flow of pointing movement. Index Terms — deictic gesture, pointing, learning, image feature, infant development I