Colour-based gesture recognition for American Sign Language via Hidden Markov Models

We present a new approach to gesture recognition for use in a signlanguage learning environment. This method utilizes inexpensivecloth gloves to alleviate the difficulty of hand detection and to allowfor feature creation. Salient colours identify the glove base andfingertip markers, which are then used to extract a hand centroidand a convex hull describing the fingertips for each hand. A HiddenMarkov Model is created for each sign, as well as an additionalthreshold model created from all signs. When a candidate sign isperformed, the sign of the HMM that produces the greatest likelihoodis matched, provided it also exceeds the threshold modellikelihood. Isolated recognition testing of the training library indicated76% accuracy, and continuous recognition testing showed60% accuracy