Multi-Feature Max-Margin Hierarchical Bayesian Model for Action Recognition

In this paper, a multi-feature max-margin hierarchical Bayesian model (M3HBM) is proposed for action recogni-tion. Different from existing methods which separate repre-sentation and classification into two steps, M3HBM joint-ly learns a high-level representation by combining a hier-archical generative model (HGM) and discriminative max-margin classifiers in a unified Bayesian framework. Specif-ically, HGM is proposed to represent actions by distribu-tions over latent spatial temporal patterns (STPs) which are learned from multiple feature modalities and shared a-mong different classes. For recognition, we employ Gibbs classifiers to minimize the expected loss function based on the max-margin principle and use the classifiers as regu-larization terms of M3HBM to perform Bayeisan estimation for classifier parameters together with the learning of STP-s. In addition, multi-task learning is applied to learn the model from multiple feature modalities for different class-es. For test videos, we obtain the representations by the inference process and perform action recognition by the learned Gibbs classifiers. For the learning and inference process, we derive an efficient Gibbs sampling algorithm to solve the proposed M3HBM. Extensive experiments on several datasets demonstrate both the representation power and the classification capability of our approach for action recognition. 1