A CONDITIONAL RANDOM FIELD APPROACH FOR FACE IDENTIFICATION IN BROADCAST NEWS USING OVERLAID TEXT

We investigate the problem of face identification in broadcast programs where people names are obtained from text over-lays automatically processed with Optical Character Recog-nition (OCR) and further linked to the faces throughout the video. To solve the face-name association and propagation, we propose a novel approach that combines the positive ef-fects of two Conditional Random Field (CRF) models: a CRF for person diarization (joint temporal segmentation and asso-ciation of voices and faces) that benefit from the combination of multiple cues including as main contributions the use of identification sources (OCR appearances) and recurrent local face visual background (LFB) playing the role of a namedness feature; a second CRF for the joint identification of the per-son clusters that improves identification performance thanks to the use of further diarization statistics. Experiments con-ducted on a recent and substantial public dataset of 7 different shows demonstrate the interest and complementarity of the different modeling steps and information sources, leading to state of the art results