A sensor fusion approach for recognizing continuous human grasping sequences using hidden markov models

nique aims at teaching a robot to accomplish a task by learning from a human demonstration. In a manipulation context, rec-ognizing the demonstrator’s hand gestures, specifically when and how objects are grasped, plays a significant role. Here, a system is presented that uses both hand shape and contact-point information obtained from a data glove and tactile sensors to recognize continuous human-grasp sequences. The sensor fusion, grasp classification, and task segmentation are made by a hidden Markov model recognizer. Twelve different grasp types from a general, task-independent taxonomy are recognized. An accuracy of up to 95 % could be achieved for a multiple-user system. Index Terms—Hidden Markov models (HMMs), Programming by Demonstration (PbD), sensor fusion, user interfaces