Gestural Interaction Using Feature Classification

Abstract. This paper describes our ongoing research work on deviceless interac-tion using hand gesture recognition with a calibrated stereo system. Video-based interaction is one of the most intuitive kinds of Human-Computer-Interaction with Virtual-Reality applications due to the fact that users are not wired to a com-puter. If interaction with three-dimensional environments is considered, pointing, grabbing and releasing are the most intuitive gestures used by humans. This pa-per describes our video-based gesture recognition system that observes the user in front of a large displaying screen, identifying three different hand gestures in real time using 2D feature classification and determines 3D information like the 3D position of the user’s hand or the pointing direction if performed. Different sce-nario applications like a virtual chess game against the computer or an industrial scenario have been developed and tested. To estimate the possible count of dis-tinguishable gestures a sign language recognition application has been developed and tested using a single uncalibrated camera only.