Real-Time Sign Language Detection using Human Pose Estimation

We propose a lightweight real-time sign language detectionmodel, as we identify the need for such a case in videoconferencing. Weextract optical flow features based on human pose estimation and, usinga linear classifier, show these features are meaningful with an accuracyof 80%, evaluated on the Public DGS Corpus. Using a recurrent modeldirectly on the input, we see improvements of up to 91% accuracy,whilestill working under 4ms. We describe a demo application to signlanguagedetection in the browser in order to demonstrate its usage possibility invideoconferencing applications