Synergistic clustering of image and segment descriptors for unsupervised scene understanding

With the advent of cheap, high fidelity, digital imaging systems, the quantity and rate of generation of visual data can dramatically outpace a humans ability to label or an-notate it. In these situations there is scope for the use of unsupervised approaches that can model these datasets and automatically summarise their content. To this end, we present a totally unsupervised, and annotation-less, model for scene understanding. This model can simultaneously cluster whole-image and segment descriptors, thereby form-ing an unsupervised model of scenes and objects. We show that this model outperforms other unsupervised models that can only cluster one source of information (image or seg-ment) at once. We are able to compare unsupervised and su-pervised techniques using standard measures derived from confusion matrices and contingency tables. This shows that our unsupervised model is competitive with current super-vised and weakly-supervised models for scene understand-ing on standard datasets. We also demonstrate our model operating on a dataset with more than 100,000 images col-lected by an autonomous underwater vehicle. 1