Real time hand gesture recognition including hand segmentation and tracking

Abstract. In this paper we present a system that performs automatic gesture recognition. The system consists of two main components: (i) A unified tech-nique for segmentation and tracking of face and hands using a skin detection algorithm along with handling occlusion between skin objects to keep track of the status of the occluded parts. This is realized by combining 3 useful features, namely, color, motion and position. (ii) A static and dynamic gesture recogni-tion system. Static gesture recognition is achieved using a robust hand shape classification, based on PCA subspaces, that is invariant to scale along with small translation and rotation transformations. Combining hand shape classifi-cation with position information and using DHMMs allows us to accomplish dynamic gesture recognition.