Evaluating the Efficiency of CBAM-ResNet by using Malaysian Sign Language (BIM)

Malaysian Sign Language has been widely used by deaf-mutes in our nation for communication since created in 1998. However, most Malaysians do not understand this sign language and have difficulties in communicating with deaf-mutes. This had discouraged deafmutes and made them felt helpless when not being understood. Issues such as unfair treatment in education and work sometimes happened. As machine learning and computer vision domain developed, these technologies provided alternates in bridging the communication gap between deaf-mutes and others. This research introduced a competitive CNNs based neural network, namely CBAM-ResNet to prove its efficiency in Malaysian Sign language recognition. A dataset consists of 2071 videos for 19 dynamic signs was built to provide instances for network training. Two different experiments were conducted for static and dynamic signs using CBAM-2DResNet and CBAM-3DResNet respectively by two CBAM integration methods, which known as ‘Within blocks’ and ‘Before classifier’. Performance metrics such as accuracy, loss, precision, recall, F1-score, confusion matrix, and training time were taken to evaluate models’ efficiency. The results showed that all CBAM-ResNet models implemented had good performances in image and video recognition tasks, with recognition rates of over 90 % with little variation. CBAM-ResNet ‘Before Classifier’ is more efficient than ‘Within blocks’ models of CBAM-ResNet in terms of various performance metrics with shorter training time required. The best trained CBAM-2DResNet was chosen to build a realtime sign recognition application based on the image recognition technique. All experiment results indicated the CBAM-ResNet ‘Before classifier’ effectiveness in recognizing Malaysian Sign Language and it’s worth of future research