A multimodal approach to initialisation for top-down speaker diarization of television shows

This paper presents a new multimodal approach to speaker diarization of TV show data. We hypothesize that the intra-speaker variation in visual information might be less than that in the corresponding acoustic information and therefore might be better suited to the task of speaker model initialisa-tion. This is an acknowledged weakness of the computation-ally efficient top-down approach to speaker diarization that is used here. Experimental results show that a recently pro-posed approach to purification and the new multimodal ap-proach to initialisation together deliver 22 % and 17 % relative improvements in diarization performance over the baseline system on independent development and evaluation datasets respectively. 1