COMPLEX EVENT RECOGNITION BY LATENT TEMPORAL MODELS OF CONCEPTS

Complex event recognition is an expanding research area aim-ing to recognize entities of high-level semantics in videos. Typical approaches exploit the so-called “bags ” of spatio-temporal features such as STIP, ISA and DTF-HOG; yet, more recently, the notion of concept has emerged as an al-ternative, intermediate representation with greater descriptive power, and “bags of concepts ” have been used for recognition. In this paper we argue that concepts in an event tend to artic-ulate over a discernible temporal structure and we exploit a temporal model using the scores of concept detectors as mea-surements. In addition, we propose several heuristics to im-prove the initialization of the model’s latent states and take advantage of the time-sparsity of the concepts. Experimental results on videos from the challenging TRECVID MED 2012 dataset show that the proposed approach achieves an improve-ment in average precision of 8.92 % over comparable bags of concepts, thus validating the use of temporal structure over concepts for complex event recognition. 1. INTRODUCTION AND RELATE