Harmonium models for semantic video representation and classification

Accurate and efficient video classification demands the fusion of multimodal information and the use of interme-diate representations. Combining the two ideas into the one framework, we propose a probabilistic approach for video classification using intermediate semantic repre-sentations derived from multi-modal features. Based on a class of bipartite undirected graphical models named harmonium, our approach represents the video data as latent semantic topics derived by jointly modeling the transcript keywords and color-histogram features, and performs classification using these latent topics under a unified framework. We show satisfactory classification performance of our approach on a benchmark dataset as well as interesting insights into the data.