Hardware Prototyping and Algorithm Development for Endoscopic Vision Systems

Thesis (Ph.D.)--University of Washington, 2022Medical endoscopy is a key technology of (semi)surface-based imaging of human organ for diagnosis in medical screening, surgical guidance in minimally invasive surgery or tele-surgery and cancer surveillance in re-examinations. The significant role of endoscope in these applications has been strengthened over the last few decades by efforts in the following directions. Firstly, build new endoscope hardware system that functions better to acquire desired images. Secondly, construct auxiliary system to assist usage of endoscope. Thirdly, develop computational tools for automated processing and understanding of endoscope images, and furthermore, for guidance of computer-aided interventions. Based on these three directions, this thesis presents our research works that result in hardware and software prototypes in efforts to advance the technology of medical endoscopy. Innovative hardware prototypes in this thesis were designed based on the scanning fiber endoscope invented in Human Photonics Lab, University of Washington. Chapter 2 reports an innovative endoscope system nirSFE (near-infrared scanning fiber endoscope) for dental imaging, which has the advantages of easier operation due to flexible and miniature scope as well as more sensitive detection of dental decay due to deeper penetration of near-infrared light into the tooth. Chapter 3 further presents an AR-based auxiliary system for visualization and guidance for nirSFE, which can potentially be used for computer-aided support system both during training and during procedure time. Besides hardware prototyping, computational tools were also developed to lay the groundwork for 3D endoscopy in computer-assisted diagnosis and surgery and even tele-surgery in the foreseeable future. Due to limitations in the early-stage SFE hardware prototype, the software tools in this thesis were designed and tested on commercial endoscopes for easy generalization to any available endoscope system in the clinics. Chapter 4 presents a toolset for synthesis of endoscope videos and evaluation of 3D reconstruction pipeline. Chapter 5 reports the improvement on a 3D reconstruction pipeline to generate a textured 3D surface model of patient bladder using clinical videos acquired by flexible cystoscope, which enables computer-assisted diagnosis and surgery. Lastly, Chapter 6 presents a scope localization pipeline based on efficient image retrieval and camera pose recovery, which, along with the reconstructed 3D model of human organ are the two key components for tele-surgery