Naming persons in video: using the weak supervision of textual stories

In this paper, we focus on the problem of automated video annotation. We report on the application of naming faces in soap series by using the weak supervision of narrative texts that describe the events in the video and that are drafted by fans. Several unsupervised methods that operate without any manual labeling of exemplar faces, and methods that use a limited number of labeled exemplars are presented and evaluated. All methods exploit the multiple co-occurrences between faces shown in the video and names mentioned in the texts to compute the strength of the linking and reinforce this coupling by means of an Expectation Maximization algorithm. We show that the unsupervised methods attain competitive results without any prior human supervision. The results show F1 values between 80% and 86% for the recognition of the face-name pairs without any human supervision. These figures rise only slightly when a number of faces were manually labeled beforehand. The study gives insights in the benets and bottlenecks of the proposed approaches, and an error analysis results in guidelines for the choice of a certain technique.The Pham P., Deschacht K., Tuytelaars T., Moens M.-F., ''Naming persons in video: Using the weak supervision of textual stories'', Journal of visual communication and image representation, vol. 24, no. 7, pp. 944-955, October 2013.status: publishe