Dense saliency-based spatiotemporal feature points for action recognition

Several spatiotemporal feature point detectors have been recently used in video analysis for action recognition. Feature points are detected using a number of measures, namely saliency, cornerness, periodicity, motion activity etc. Each of these measures is usually intensity-based and provides a different trade-off between density and informa-tiveness. In this paper, we use saliency for feature point detection in videos and incorporate color and motion apart from intensity. Our method uses a multi-scale volumetric representation of the video and involves spatiotemporal op-erations at the voxel level. Saliency is computed by a global minimization process constrained by pure volumetric con-straints, each of them being related to an informative visual aspect, namely spatial proximity, scale and feature similar-ity (intensity, color, motion). Points are selected as the ex-trema of the saliency response and prove to balance well between density and informativeness. We provide an in-tuitive view of the detected points and visual comparisons against state-of-the-art space-time detectors. Our detec-tor outperforms them on the KTH dataset using Nearest-Neighbor classifiers and ranks among the top using differ-ent classification frameworks. Statistics and comparisons are also performed on the more difficult Hollywood Human Actions (HOHA) dataset increasing the performance com-pared to current published results. 1