Ontology-based image categorization

In this thesis, we study how semantics can improve image categorization. Previous image categorization approaches mostly neglect semantics, which has two major limitations. First, object classes have semantic overlaps. For example, “sedan” is a specific kind of “car”. However, previous approaches treat “sedan” and “car” as independent and train a classifier to distinguish them, which is unreasonable. Second, image features used for classification are unified for different object classes. But this is at odds with the human perception system, which is believed to use different features for distinct objects. For example, the features used for differentiating “sedan” from “bike” should be distinct from the features used for differentiating “sedan” from “hatchback”. In this thesis, we leverage semantic ontologies to solve the aforementioned problems. We propose a Random Forest based algorithm in which the splitting of tree nodes is first determined by semantic relations among categories. Then weak attributes are automatically learned by multiple-instance learning to capture visual similarities in a hierarchical way; i.e., different local features are learned to classify objects at different semantic levels. Overall, our approach imitates the human visual system and is more advanced than previous non-ontology based approaches. We test our approach on two fine-grained image categorization datasets. The experimental results demonstrate that our approach not only outperforms the state-of-the-art approaches but also identifies semantically meaningful visual features