Real-time Hand Gesture Recognition on Unmodified Wearable Devices

We present a machine learning technique for recognizing discrete gestures and estimating continuous 3D hand posi-tion for mobile interaction. Our multi-stage random forest pipeline jointly classifies hand shapes and regresses metric depth of the hand from a single RGB camera. Our tech-nique runs in real time on unmodified mobile devices, such as smartphones, smartwatches, and smartglasses, comple-menting existing interaction paradigms with in-air gestures. 1