Automatic Segmentation, Aggregation and Indexing of Multimodal News Information from Television and the Internet

ABSTRACT: The global diffusion of the Internet has enabled the distribution of informative content through dynamic media such as RSS feeds and video blogs. At the same time, the decreasing cost of electronic devices has increased the pervasive availability of the same informative content in the form of digital audiovisual data. This paper presents a system for the large-scale unsupervised acquisition, segmentation and indexing of TV newscasts. In particular, it discusses the principles and performance of the parts of the system dedicated to the detection and segmentation of programmes from the acquired stream. In addition to the core technology, we also introduce and discuss a novel method for assessing the results of story boundaries segmentation algorithms, based on a user-validated measurement. Due to the heterogeneity of current news distribution channels, a further innovative aspect of this paper is the description of a framework for multimodal information aggregation. The core of this framework is a cross-modal clustering process for which a novel, asymmetric similarity measure is provided. The implemented prototype uses online news articles and TV news programmes as information sources, and provides a multimodal service integrating both contributions. Experimental evaluation of the system proves the effectiveness of the method in the studied case