Visual synset: towards a higher-level visual representation

We present a higher-level visual representation, visual synset, for object categorization. The visual synset im-proves the traditional bag of words representation with bet-ter discrimination and invariance power. First, the ap-proach strengthens the inter-class discrimination power by constructing an intermediate visual descriptor, delta visual phrase, from frequently co-occurring visual word-set with similar spatial context. Second, the approach achieves bet-ter intra-class invariance power, by clustering delta visual phrases into visual synset, based their probabilistic ’seman-tics’, i.e. class probability distribution. Hence, the resulting visual synset can partially bridge the visual differences of images of same class. The tests on Caltech-101 and Pascal-VOC 05 dataset demonstrated that the proposed image rep-resentation can achieve good accuracies. 1