Extraction of spatial-temporal features for vision-based gesture recognition

One of the key problems in a vision-based gesture recognition system is the extraction of spatial-temporal features of gesturing. In this paper we propose an approach of motion-based segmentation to realize this task. We use the direct method cooperated with the robust M-estimator to estimate the affine parameters of gesturing motion, and based on the dominant motion model we extract the gesturing region, i. e. the dominant object. So the spatial-temporal features of gestures have been extracted. Finally, we directly use the dynamic time warping (DTW) method to perform matching of 12 control gestures (6 for "translation " orders, 6 for "rotation " orders). A small demonstration system has been set up to verify our method, in which we can control a panorama image viewer (set by mosaicing a sequence of standard "Garden " images) with recognized gestures instead of the 3-D mouse tool