Learning Ensembles of Potential Functions for Structured Prediction with Latent Variables

Many visual recognition tasks involve modeling vari-ables which are structurally related. Hidden conditional random fields (HCRFs) are a powerful class of models for encoding structure in weakly supervised training exam-ples. This paper presents HCRF-Boost, a novel and general framework for learning HCRFs in functional space. An al-gorithm is proposed to learn the potential functions of an HCRF as a combination of abstract nonlinear feature func-tions, expressed by regression models. Consequently, the resulting latent structured model is not restricted to tradi-tional log-linear potential functions or any explicit param-eterization. Further, functional optimization helps to avoid direct interactions with the possibly large parameter space of nonlinear models and improves efficiency. As a result, a complex and flexible ensemble method is achieved for struc-tured prediction which can be successfully used in a variety of applications. We validate the effectiveness of this method on tasks such as group activity recognition, human action recognition, and multi-instance learning of video events. 1