FINGeR: Framework for Interactive Neural-based Gesture Recognition

Abstract. For operating in real world scenarios, the recognition of human gestures must be adaptive, robust and fast. Despite the prominent use of Kinect-like range sensors for demanding visual tasks involving motion, it still remains unclear how to process depth information for efficiently extrapolating the dynamics of hand gestures. We propose a learning framework based on neural evidence for processing visual information. We first segment and extract spatiotemporal hand properties from RGB-D videos. Shape and motion features are then processed by two parallel streams of hierarchical self-organizing maps and subsequently combined for a more robust representation. We provide experimental results to show how multicue integration increases recognition rates over a single-cue approach.