Automatic recognition of Auslan finger-spelling using hidden Markov models. Undergraduate thesis

In recent years, gesture recognition has received much attention from research communities. Computer vision-based gesture recognition has many potential ap-plications in the area of human-computer interaction as well as sign language recognition. Sign languages use a combination of hand shapes, motion and loca-tions as well as facial expressions. Finger-spelling is a manual representation of alphabet letters, which is often used where there is no sign word to correspond to a spoken word. In Australia, a sign language called Auslan is used by the deaf community and and the finger-spelling letters use two handed motion, unlike the well known finger-spelling of American Sign Language (ASL) that uses static shapes. This thesis presents the Auslan Finger-spelling Recognizer (AFR) that is a real-time system capable of recognizing signs that consists of Auslan manual alphabet letters from video sequences. The AFR system has two components: the first is the feature extraction process that extracts a combination of spatial and motion features from the images. Which classifies a sequence of features using Hidden Markov Models (HMMs). Tests using a vocabulary of twenty signed words showed the system could achieve 97 % accuracy at the letter level and 88% at the word level using a finite state grammar network and embedded training