Visual Word Spatial Arrangement For Image Retrieval And Classification

We present word spatial arrangement (WSA), an approach to represent the spatial arrangement of visual words under the bag-of-visual-words model. It lies in a simple idea which encodes the relative position of visual words by splitting the image space into quadrants using each detected point as origin. WSA generates compact feature vectors and is flexible for being used for image retrieval and classification, for working with hard or soft assignment, requiring no pre/post processing for spatial verification. Experiments in the retrieval scenario show the superiority of WSA in relation to Spatial Pyramids. Experiments in the classification scenario show a reasonable compromise between those methods, with Spatial Pyramids generating larger feature vectors, while WSA provides adequate performance with much more compact features. As WSA encodes only the spatial information of visual words and not their frequency of occurrence, the results indicate the importance of such information for visual categorization. © 2013 Elsevier Ltd.472705720Sivic, J., Zisserman, A., Video google: A text retrieval approach to object matching in videos (2003) International Conference on Computer Vision, 2, pp. 1470-1477Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., Kadir, T., Gool, L., A comparison of affine region detectors (2005) International Journal of Computer Vision, 65, pp. 43-72Van De Sande, K.E.A., Gevers, T., Snoek, C.G.M., Evaluating color descriptors for object and scene recognition (2010) Transactions on Pattern Analysis and Machine Intelligence, 32 (9), pp. 1582-1596Lowe, D.G., Distinctive image features from scale-invariant keypoints (2004) International Journal of Computer Vision, 60 (2), pp. 91-110Fei-Fei, L., Fergus, R., Perona, P., Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories (2004) Conference on Computer Vision and Pattern Recognition Workshop, 12, p. 178Andreopoulos, A., Tsotsos, J.K., 50 years of object recognition: Directions forward (2013) Computer Vision and Image Understanding, 117 (8), pp. 827-891Penatti, O.A.B., Valle, E., Torres, R.D.S., Encoding spatial arrangement of visual words (2011) Iberoamerican Congress on Pattern Recognition, 7042, pp. 240-247Hoàng, N.V., Gouet-Brunet, V., Rukoz, M., Manouvrier, M., Embedding spatial information into image content description for scene retrieval (2010) Pattern Recognition, 43, pp. 3013-3024Lazebnik, S., Schmid, C., Ponce, J., Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories (2006) Conference on Computer Vision and Pattern Recognition, 2, pp. 2169-2178Feng, J., Ni, B., Tian, Q., Yan, S., Geometric lp-norm feature pooling for image classification (2011) Conference on Computer Vision and Pattern Recognition, pp. 2609-2704Cao, Y., Wang, C., Li, Z., Zhang, L., Zhang, L., Spatial-bag-of-features (2010) Conference on Computer Vision and Pattern Recognition, pp. 3352-3359Zhou, W., Lu, Y., Li, H., Song, Y., Tian, Q., Spatial coding for large scale partial-duplicate web image search (2010) International Conference on Multimedia, pp. 511-520Jégou, H., Douze, M., Schmid, C., Improving bag-of-features for large scale image search (2010) International Journal of Computer Vision, 87, pp. 316-336Weber, R., Schek, H., Blott, S., A quantitative analysis and performance study for similarity-search methods in high-dimensional spaces (1998) International Conference on Very Large Data Bases, pp. 194-205Traina, J.C., Traina, A., Faloutsos, C., Seeger, B., Fast indexing and visualization of metric data sets using slim-trees (2002) Transactions on Knowledge and Data Engineering, 14 (2), pp. 244-260Kang, H., Hebert, M., Kanade, T., Image matching with distinctive visual vocabulary (2011) IEEE Workshop on Applications of Computer Vision, pp. 402-409Van Gemert, J.C., Veenman, C.J., Smeulders, A.W.M., Geusebroek, J.-M., Visual word ambiguity (2010) Transactions on Pattern Analysis and Machine Intelligence, 32, pp. 1271-1283Philbin, J., Chum, O., Isard, M., Sivic, J., Zisserman, A., Lost in quantization: Improving particular object retrieval in large scale image databases (2008) Conference on Computer Vision and Pattern Recognition, pp. 1-8Liu, L., Wang, L., Liu, X., In defense of soft-assignment coding (2011) International Conference on Computer Vision, pp. 1-8Penatti, O.A.B., Torres, R.D.S., Eva-an evaluation tool for comparing descriptors in content-based image retrieval tasks (2010) International Conference on Multimedia Information Retrieval, pp. 413-416Viitaniemi, V., Laaksonen, J., Experiments on selection of codebooks for local image feature histograms (2008) International Conference on Visual Information Systems: Web-Based Visual Information Search and Management, pp. 126-137Jurie, F., Triggs, B., Creating efficient codebooks for visual recognition (2005) International Conference on Computer Vision, 1, pp. 604-610Boureau, Y.-L., Bach, F., Lecun, Y., Ponce, J., Learning mid-level features for recognition (2010) Conference on Computer Vision and Pattern Recognition, pp. 2559-2566Jegou, H., Douze, M., Schmid, C., Hamming embedding and weak geometric consistency for large scale image search (2008) European Conference on Computer Vision. Part i, 5302, pp. 304-317Avila, S., Bossa: Extended bow formalism for image classification (2011) International Conference on Image Processing, pp. 2966-2969Perronnin, F., Improving the fisher kernel for large-scale image classification (2010) European Conference on Computer Vision, 6314, pp. 143-156Mbanya, E., Gerke, S., Ndjiki-Nya, P., Spatial codebooks for image categorization (2011) International Conference on Multimedia Retrieval, pp. 501-507Karaman, S., Multi-layer local graph words for object recognition (2012) Advances in Multimedia Modeling, 7131, pp. 29-39Smeulders, A.W.M., Worring, M., Santini, S., Gupta, A., Jain, R., Content-based image retrieval at the end of the early years (2000) Transactions on Pattern Analysis and Machine Intelligence, 22 (12), pp. 1349-1380Huang, J., Kumar, S.R., Mitra, M., Zhu, W., Zabih, R., Image indexing using color correlograms (1997) Conference on Computer Vision and Pattern Recognition, p. 762Pass, G., Zabih, R., Miller, J., Comparing images using color coherence vectors (1996) ACM Multimedia, pp. 65-73Zhou, W., Li, H., Lu, Y., Tian, Q., Large scale image search with geometric coding (2011) ACM Multimedia, pp. 1349-1352Sivic, J., Russell, B.C., Efros, A.A., Zisserman, A., Freeman, W.T., Discovering objects and their location in images (2005) International Conference on Computer Vision, 1, pp. 370-377Savarese, S., Winn, J., Criminisi, A., Discriminative object class models of appearance and shape by correlations (2006) Conference on Computer Vision and Pattern Recognition, 2, pp. 2033-2040Jegou, H., Douze, M., Schmid, C., Perez, P., Aggregating local descriptors into a compact image representation (2010) Conference on Computer Vision and Pattern Recognition, pp. 3304-3311Torralba, A., Efros, A.A., Unbiased look at dataset bias (2011) Conference on Computer Vision and Pattern Recognition, pp. 1521-152