Multimodal registration of optical microscopic and infrared spectroscopic images from different tissue sections: An application to colon cancer

Fourier transform infrared (FTIR) spectroscopic images provide rich information of the biochemical tissue composition that can be analyzed together with other microscopy modalities to perform objective pathological diagnoses. Hematoxylin and Eosin (H&E) stained images are the reference images that pathologists use to make a final diagnosis of most diseases, such as cancer. Therefore, H&E images may be the most interesting imaging modality to be fused with FTIR images. Unfortunately, H&E stain introduces severe confounding artifacts in the FTIR spectra. Thus, in repeatable studies different slices of tissue must be used to acquire images for each imaging modality, which must be aligned so that the different regions of tissue spatially match. The main objective of this manuscript is to establish a complete pipeline where the two types of images (H&E and FTIR) from different tissue sections are aligned or registered. The proposed automatic framework starts by obtaining grayscale images from both FTIR raw data and H&E images where analogous anatomical structures are easily distinguishable. In the first alignment step, a feature-based registration produces a fast coarse rigid alignment by using the Scale Invariant Feature Transform (SIFT) algorithm to automatically find and match relevant keypoints in both grayscale images. Due to the spatial variability between samples, different combinations of SIFT parameters are explored and the best combination is selected through the maximization of a similarity measure between the aligned images. In the second alignment step, an intensity-based registration refines the initial alignment and compensates for the local spatial differences between the tissue sections by iteratively estimating a non-rigid transformation. This methodology was used to register 47 colon samples from three different pathological groups (16 normal, 16 intermediate and 15 tumoral) with good overall results, which were quantitatively evaluated for both registration steps. In the first rigid alignment step, the global median of difference in positioning compared to a manual registration was under 1 pixel. In the second registration step, the global median gain in mutual information between the registered images was 0.125 bits. In contrast to existing approaches, the proposed method does not need a prior segmentation step that may introduce errors and reduce the spatial information content, which is crucial when different sections of tissue are used. It can improve the accuracy to combine the spatial information extracted from both the traditional H&E stained images and the emerging FTIR spectroscopy