Hierarchical Part Matching for Fine-Grained Visual Categorization

As a special topic in computer vision, fine-grained visual categorization (FGVC) has been attracting growing atten-tion these years. Different with traditional image classifi-cation tasks in which objects have large inter-class varia-tion, the visual concepts in the fine-grained datasets, such as hundreds of bird species, often have very similar seman-tics. Due to the large inter-class similarity, it is very difficult to classify the objects without locating really discriminative features, therefore it becomes more important for the algo-rithm to make full use of the part information in order to train a robust model. In this paper, we propose a powerful flowchart named Hierarchical Part Matching (HPM) to cope with fine-grained classification tasks. We extend the Bag-of-Features (BoF) model by introducing several novel modules to inte-grate into image representation, including foreground in-ference and segmentation, Hierarchical Structure Learn-ing (HSL), and Geometric Phrase Pooling (GPP). We ver-ify in experiments that our algorithm achieves the state-of-the-art classification accuracy in the Caltech-UCSD-Birds-200-2011 dataset by making full use of the ground-truth part annotations. 1