Patch-Based Nonlinear Image Registration For Gigapixel Whole Slide Images

Introduction/ Background Image Registration of whole slide histology images allows the fusion of fine-grained information – like different immunohistochemical stains – from neighboring tissue slides. Traditionally, pathologists fuse this information by looking subsequently at one slide at a time. If the slides are digitized and accurately aligned at cell-level, automatic analysis can be used to ease the pathologist’s work. However, the size of those images exceeds the memory capacity of regular computers, preventing the application of established image registration methods at a high magnification. Aims An accurate and automatic alignment helps the pathologist to analyze the combination of different antibodies without memorizing multiple slides. For some applications, cell-level accuracy is needed. This also enables automatic image analysis to take advantage of multislide information. Methods We extend available registration methods by using image data at fine spatial resolution. However, this data is not simultaneously globally available due to the computer’s memory restrictions. Typical approaches either reduce the amount of data to be processed by downsampling or partition the data into smaller chunks to be processed separately. We combine the patch based approach with an elastic deformation model to obtain a global registration result. Our method first registers the complete images on a low resolution with a nonlinear deformation model and later refines this result on patches by using a second nonlinear registration on each patch. The deformation information on the individual patches can be contradictory and needs to be combined into one global model. As an extension to our previous work, the global solution is computed by minimizing an energy function that preserves the patch-wise deformation and establishes global smoothness. The NGF distance measure is used to handle multi-stain images. Results The method will be applied to whole slide image pairs. The alignment of corresponding structures will be measured by comparing manual segmentations from neighboring slides. First results show an improvement of the registration accuracy compared to the low-resolution nonlinear registration.