Combining the Right Features for Complex Event Recognition

In this paper, we tackle the problem of combining fea-tures extracted from video for complex event recognition. Feature combination is an especially relevant task in video data, as there are many features we can extract, rang-ing from image features computed from individual frames to video features that take temporal information into ac-count. To combine features effectively, we propose a method that is able to be selective of different subsets of features, as some features or feature combinations may be unin-formative for certain classes. We introduce a hierarchi-cal method for combining features based on the AND/OR graph structure, where nodes in the graph represent com-binations of different sets of features. Our method auto-matically learns the structure of the AND/OR graph using score-based structure learning, and we introduce an infer-ence procedure that is able to efficiently compute structure scores. We present promising results and analysis on th