Robust South African sign language gesture recognition using hand motion and shape

Magister Scientiae - MScResearch has shown that five fundamental parameters are required to recognize any sign language gesture: hand shape, hand motion, hand location, hand orientation and facial expressions. The South African Sign Language (SASL) research group at the University of the Western Cape (UWC) has created several systems to recognize sign language gestures using single parameters. These systems are, however, limited to a vocabulary size of 20 – 23 signs, beyond which the recognition accuracy is expected to decrease. The first aim of this research is to investigate the use of two parameters – hand motion and hand shape – to recognise a larger vocabulary of SASL gestures at a high accuracy. Also, the majority of related work in the field of sign language gesture recognition using these two parameters makes use of Hidden Markov Models (HMMs) to classify gestures. Hidden Markov Support Vector Machines (HM-SVMs) are a relatively new technique that make use of Support Vector Machines (SVMs) to simulate the functions of HMMs. Research indicates that HM-SVMs may perform better than HMMs in some applications. To our knowledge, they have not been applied to the field of sign language gesture recognition. This research compares the use of these two techniques in the context of SASL gesture recognition. The results indicate that, using two parameters results in a 15% increase in accuracy over the use of a single parameter. Also, it is shown that HM-SVMs are a more accurate technique than HMMs, generally performing better or at least as good as HMMs