H: Color and texture based segmentation of molecular pathology images using HSOMS

Prostate cancer is the most common cancer among men, excluding skin cancer. It is diagnosed by histopathology in-terpretation of Hematoxylin and Eosin (H&E)-stained tissue sections. Gland and nuclei distributions vary with the disease grade, and the morphological features vary with the advance of cancer. A tissue microarray with known disease stages can be used to enable efficient pathology slide image analysis. We focus on an intuitive approach for segmenting such images, using the Hierarchical Self-Organizing Map (HSOM). Our approach introduces the use of unsupervised clustering using both color and texture features, and the use of unsupervised color merging outside of the HSOM framework. The HSOM was applied to segment 109 tissues composed of four tissue clusters: glands, epithelia, stroma, and nuclei. These seg-mentations were compared with the results of an EM Gaus-sian clustering algorithm. The proposed method confirms that the self-learning ability and adaptability of the HSOM, cou-pled with the information fusion mechanism of the hierarchi-cal network, leads to superior segmentation results for tissue images. Index Terms — Molecular pathology, Hematoxylin and Eosin staining (H&E), Hierarchical selforganizing map