Malaysian Sign Language Recognition Using Skeleton Data with Neural Network

This paper proposed the Backpropagation Neural Network (BNN) for recognition of Malaysian Sign Language. The image acquisition is done with the help of skeletal tracking using a Kinect camera. The data samples tested there are 15 dynamic signs taken from the Malaysian Sign Language (MySL). Pre-processing in this study was based on skeleton tracking the joints, to produce 3D coordinates X, Y, Z. The sample of Data 3D coordinates X, Y, and Z is taken a value relative to the Spine and head. The feature extraction is done by normalizing the position and size of the user, by taking eight out of 20 joints contributed in identifying the movement with the hand, left hand, right hand, left wrist, right wrist, left elbow, right elbow, Spine, Head. We use spherical coordinate conversion process and grouping frame used mean function. Two different approaches are evaluated: (1) the Cartesian coordinates (X, Y, and Z) of the six used joints and number of group's frames 14. The classifier used is the Neural Network (Cartesian + ANN); (2) the spherical coordinates (X, Y, and Z) and number of group's frame 14. The classifier used is Neural Network (Spherical+ANN). The experimental results of the Cartesian coordinate, and Artificial Neural Network (ANN) is the best of the neuron hidden layer 150, the best average recognition rate 80.54% with ± 2.07%, maximum 82.61% and minimum 78.47%. The increase at the recognition rate of Cartesian Coordinate+ANN with Spherical coordinates is 12.93%