Temporal Patterns in Social Media Streams: Theme Discovery and Evolution Using Joint Analysis of Content and Context

Online social networking sites such as Flickr, YouTube and Facebook provide a diverse range of functionalities that foster online communities to create and share media content. In particular, Flickr groups are increasingly used to aggregate and share photos about a wide array of topics or themes. Unlike erstwhile photo repositories where images used to be organized with respect to static topics, the photo sharing process as in Flickr often results in miscellaneous time-evolving social and visual patterns. Characterizing such time-evolving patterns can enrich media exploring experience in a social media repository. In this paper, we propose a novel framework that characterizes distinct time-evolving patterns of group photo streams. We use a non-negative joint matrix factorization approach to incorporate image content features and contextual information, including associated tags, photo owners and post times. In our framework, we consider a group as a mixture of themes – each theme exhibits similar patterns of image content and context. The theme extraction is formulated as an optimization problem where the objective is to best explain the observed image content features and associations with tags, users and times. Extensive experiments on a Flickr dataset suggest that our approach is able to extract meaningful evolutionary patterns from group photo streams. We evaluate our method through a tag prediction task. Our prediction results outperform baseline methods, which indicate the utility of our theme based joint analysis.