IEEE TRANSACTIONS ON MULTIMEDIA TO APPEAR 1 ARGOS: Automatically extracting Repeating Objects from multimedia Streams

Many media streams consist of distinct objects that repeat. For example broadcast television and radio signals contain advertisements, call sign jingles, songs and even whole programs that repeat. The problem we address is to explicitly identify the underlying structure in repetitive streams and de-construct them into their component objects. Our algorithm exploits dimension reduction techniques on the audio portion of a multimedia stream to make search and buffering feasible. Our architecture assumes no a priori knowledge of the streams, and does not require that the repeating objects be known. Everything the system needs, including the position and duration of the repeating objects, is learned on the fly. We demonstrate that it is perfectly feasible to identify in realtime repeating objects that occur days or even weeks apart in audio or video streams. Both the compute and buffering requirements are comfortably within reach for a basic desktop computer. We outline the algorithms, enumerate several applications and present results from real broadcast streams. Index Terms Multimedia, repeats, low dimension representation, segmentation, audio fingerprin