Clustering Social Event Images using Kernel Canonical Correlation Analysis

Sharing user experiences in form of photographs, tweets, text, audio and/or video has become commonplace in so-cial networking websites. Browsing through large collec-tions of social multimedia remains a cumbersome task. It requires a user to initiate textual search query and manu-ally go through a list of resulting images to find relevant in-formation. We propose an automatic clustering algorithm, which, given a large collection of images, groups them into clusters of different events using the image features and related metadata. We formulate this problem as a ker-nel canonical correlation clustering problem in which data samples from different modalities or ‘views ’ are projected to a space where correlations between the samples ’ projec-tions are maximized. Our approach enables us to learn a semantic representation of potentially uncorrelated feature sets and this representation is clustered to give unique so-cial events. Furthermore, we leverage the rich information associated with each uploaded image (such as usernames, dates/timestamps, etc.) and empirically determine which combination of feature sets yields the best clustering score for a dataset of 100,000 images. 1