Virtual staining of slide-free microscopy images with deep learning

Histopathology is the study of tissue to look for disease. In the context of clinical medicine, it involves the microscopic examination of patient tissue samples. The standard histopathology workflow can take several hours or even days, due to the cumbersome tissue processing steps needed. This long timeline makes the standard workflow ill-suited for time-sensitive applications such as intraoperative surgical guidance. Slide-free microscopy (SFM) techniques allow for imaging of fresh tissue samples without the need for time-consuming tissue processing steps, but they often produce images that do not closely resemble conventional histology and thus may be harder to interpret by pathologists. Therefore, the present work explored the use of deep learning to convert SFM images so as to resemble hematoxylin and eosin (H&E) stained slides. This process is referred to as microscopy modality conversion or virtual staining. The dissertation focuses on virtual staining applied to three SFM methods: microscopy withultraviolet surface excitation (MUSE), fluorescence-imitating brightfield imaging (FIBI), and quantitative oblique back illumination microscopy (qOBM).In this work, three unpaired image-to-image translation algorithms were evaluated for MUSE-to-H&E conversion, and it was concluded that an unpaired image-to image translation algorithm based on CycleGAN (cycle-consistent generative adversarial network) proved to be the methodology that performed best. FIBI, a novel SFM technique, was shown to have significant advantages when deployed for rapid histology. CycleGANs were also developed for FIBI-to-H&E conversion, and we demonstratedclinical utility of FIBI-to-H&E conversion in a preliminary dermatopathologist validation study. CycleGANs were also applied to provide H&E-like appearance for the monochrome qOBM images; performance was validated using a neural network classifier test and a user study with neuropathologists. Overall, the results of this work point to the general effectiveness of CycleGANs for SFM virtual staining.This work also explored important considerations such as impact of data quality and curation on results and usefulness of transfer learning. Current limitations and future directions of virtual staining research are also discussed, such as a need for improved evaluation tools of virtual staining methods and the development of diagnostic AI with SFM-enabled by virtual staining. The virtual staining techniques explored will hopefully enable novel, alternative workflows in histopathology that utilize SFM, potentially saving time, labor, and costs in cancer screening, treatment guidance, and more