Medical Image Retrieval Using Visual and Semantic Features

Abstract — Medical images are being digitized and the medical databases are rapidly growing. These images are used in academics, diagnoses, and hospitals for planning treatment. Data mining techniques are applied to medical images, for a quick diagnosis. Thus, the technique of Content-based Medical Image Retrieval (CBMIR) emerges as the times require. For medical image retrieval, current CBMIR is not sufficient to capture the semantic content of an image and difficult to provide good results according to the predefined categories in the medical domain by using less medical knowledge. In this paper, the retrieval system is a combination of low-level image feature and high-level semantics and it includes three main parts: In the first part, the low-level fusion visual features are extracted based on intensity, texture, and their extended versions. Secondly, a set of disjoint semantic tokens with appearance in lung CT images is selected to define a vocabulary based on medical knowledge representation. Finally, a mapping is investigated to associate low-level visual image features with their high-level semantics. In this paper a mapping modelling of visual feature and knowledge representation is presented to approach for medical image retrieval. One important contribution of this paper is the use of physicians defined linguistic variables closely related to known pathologies. This framework could be the foundation of building a novel and flexible model for diagnostic medical image retrieval that uses physician-defined semantics. Keywords- Medical image retrieval; low-level features; knowledge representation; semantic Features