Gesture-based programming for robotics: human-augmented software adaptation

Gesture-Based Programming is a paradigm for programming robots by human demonstration in which the human demonstrator directs the self-adaptation of executable software. The goal is to provide a more natural environment for the user as programmer and to generate more complete and successful programs by focusing on task experts rather than programming experts. We call the paradigm “gesture-based ” because we try to enable the system to capture, in real-time, the intention behind the demonstra-tor’s fleeting, context-dependent hand motions, contact conditions, finger poses, and even cryptic utterances in order to recon-figure itself. The system is self-adaptive in the sense that knowledge of previously acquired skills (sensorimotor expertise) is retained by the system and this knowledge facilitates the interpretation of the gestures during training and then provides feed-back control during run-time. 1