COMBINING METADATA AND CONTEXT INFORMATION IN ANNOTATING PERSONAL MEDIA

The proliferation of digital cameras, including those with mobile phones, has resulted in enormous volumes of pho-tographs and video in personal media. To make these re-sources retrievable, we need to be able to annotate 4W ’s – when, where, who and what. While time and rough loca-tion via GPS are generally available, the annotation of who, what and exact where remains challenging. In addition to visual analysis, we must make use of the available metadata and contextual-level information to extract such informa-tion. This paper focuses on the “who ” annotation problem. By assuming that personal media is mostly about the chron-icles of a person, we can make use of patterns of person’s social activities to supplement the results of face detection and recognition. By assuming that the same groups of peo-ple tend to appear in same events, and wear the same cloths within a short duration and nearby places, we can bring in the body context and social network information to esti-mate the person’s presence in the photos and other exam-ples of the same recognized persons. The combination of face detection/recognition, body context and social network has resulted in an effective system for person annotation in personal media. Experiments on a photo album containing over 1500 photos demonstrate that our approach is effective. 1