Eastman Kodak Company

We first propose a new spatio-temporal context distri-bution feature of interest points for human action recog-nition. Each action video is expressed as a set of relative XYT coordinates between pairwise interest points in a lo-cal region. We learn a global GMM (referred to as Uni-versal Background Model, UBM) using the relative coordi-nate features from all the training videos, and then repre-sent each video as the normalized parameters of a video-specific GMM adapted from the global GMM. In order to capture the spatio-temporal relationships at different lev-els, multiple GMMs are utilized to describe the context dis-tributions of interest points over multi-scale local regions. To describe the appearance information of an action video, we also propose to use GMM to characterize the distribu-tion of local appearance features from the cuboids centered around the interest points. Accordingly, an action video can be represented by two types of distribution features: 1) multiple GMM distributions of spatio-temporal context; 2) GMM distribution of local video appearance. To effectively fuse these two types of heterogeneous and complementary distribution features, we additionally propose a new learn-ing algorithm, called Multiple Kernel Learning with Aug-mented Features (AFMKL), to learn an adapted classifier based on multiple kernels and the pre-learned classifiers of other action classes. Extensive experiments on KTH, multi-view IXMAS and complex UCF sports datasets demonstrate that our method generally achieves higher recognition ac-curacy than other state-of-the-art methods. 1