People Indexing Using Audio and Video Segmentations

Abstract: In this paper, we review and propose methods for audio speaker segmentation, video costume segmentation and the automatic association between each voice and the images containing the corresponding visual person. This association can be used as a preprocessing step for existing applications like person identification systems. The first step consists in fusing, without any a priori knowledge, the two indexes produced by audio and video segmentations, in order to make the information brought by each of them more robust. Evaluation is done on a corpus composed of French TV broadcasts. If both audio and video streams are correctly segmented, this automatic association yields excellent results. When the two streams are oversegmented, our system permits to detect the main people in terms of the duration of their appearance