A hybrid stochasticconnectionist approachto gesture recognition

andreac @ informatik.tu-ilmenau.de In this paper an architecture for the recognition of d)i-namic gestures is described. The system implemented is de-signed to take a sequence of images and to assign it to one of a number of discrete classes where each of them corre-sponds to a gesture from a predefined vocabulary. The classijication task is broken down into an initial pre-processing stage following by a mapping from the prepro-cessed input variables to an output variable representing the class label. The preprocessing stage consists in the extraction of one translation and scale invariant feature vector from each image of the sequence. Further we uti-lize a hybrid combination of Kohonen Self Organizing Map (SOM) and discrete Hidden Markov Models (DHMM) for mapping an ordered sequence of feature vectors to one ges-ture category. We create one DHMM for each movement to be detected. In the learning phase the SOM is used to cluster the fea-ture vector space. After the self- organizing process each codebook is quantized into a symbol. Every symbol se-quence underlying a given movement is finally used to train the corresponding Markov model by means of the non dis-criminative Baum- Welch algorithm, aiming at maximizing the probability of the samples given the model at hand. In the recognition phase the SOM transforms any input image sequence into one symbol sequence which is subse-quently fed into a system of DHMMs. The gesture associ-ated with the model which best matches the observed sym-bol sequence is chosen as the recognized movement. Preliminary experiments with our baseline system achieved a recognition accuracy of about 82%. The data was gathered from four people pelforming five repetitions of each of five gestures from a predefined vocabulary. The system uses input from a monocular color video camera, is user-independent but not real-time. 1