3-D canonical pose estimation and abnormal gait recognition with a single RGB-D camera

Assistive robots play an important role in improving the quality of life of patients at home. Among all the monitoring tasks, gait disorders are prevalent in elderly and people with neurological conditions and this increases the risk of fall. Therefore, the development of mobile systems for gait monitoring at home in normal living conditions is important. Here, we present a mobile system that is able to track humans and analyze their gait in canonical coordinates based on a single RGB-D camera. First, view-invariant three-dimensional (3-D) lower limb pose estimation is achieved by fusing information from depth images along with 2-D joints derived in RGB images. Next, both the 6-D camera pose and the 3-D lower limb skeleton are real-time tracked in a canonical coordinate system based on simultaneously localization and mapping (SLAM). A mask-based strategy is exploited to improve the re-localization of the SLAM in dynamic environments. Abnormal gait is detected by using the support vector machine and the bidirectional long-short term memory network with respect to a set of extracted gait features. To evaluate the robustness of the system, we collected multi-cameras, ground truth data from 16 healthy volunteers performing 6 gait patterns that mimic common gait abnormalities. The experiment results demonstrate that our proposed system can achieve good lower limb pose estimation and superior recognition accuracy compared to previous abnormal gait detection methods