Detecting Very Small and Sparse Lesions in Medical Images

We have fully witnessed the rise of Convolutional Neural Networks (CNNs). They have succeeded in different intellectual tasks in different general domains such as Computer Vision (CV), Natural Language Processing (NLP), Automatic Speech Recognition (ASR), etc. CNNs' huge achievements convey their potentials in the medical and health domain. In this dissertation, we will focus on detecting small and sparse lesions in medical images. Lesions are damages and abnormal tissues in the human body. Many of them are early manifestations of fatal diseases such as cancers and tuberculosis. Thus, detecting lesions in their early stages is associated with increasing the cure rate and survival rate. Lesions at early stages, in general, are very sparse and small compared with the retrieval space (2D and 3D). For instance, a pulmonary nodule (also referred to "coin lesion") can be of a diameter smaller than 5 mm while the whole Computed Tomography (CT) scans at chest would normally be larger than 20,000 cm3 (volume ratio is less than 1/100,000). Detecting these small lesions is challenging. In the meantime, people have to strike the balance between recall and precision. Another concern would be the prohibitive GPU memory consumption when dealing with 3D images. One direct consequence is that we cannot leverage very deep architecture in our task. We devise a special end-to-end 3D Aggregated Faster R-CNN for the general lesion detection purpose. This Aggregated Faster-RCNN is equipped with a feature aggregation and local magnification operations. To further improve the model as a whole, we devise an adaptive Focal Loss. We achieve the best single-model FROC performance on LUNA16 and DeepLesion dataset with a systematically higher inference speed. Furthermore, we introduce an anchor-free design to improve the generalizing ability of our approach. This anchor-free idea is also extended to the 2D colorectal polyp detection task. In this task, we also achieve state-of-the-art performance while maintaining the real-time response. We also investigate the skip connections in the building blocks in the context of generative models. We demonstrate that solely adding skip connections in back-prop can also be helpful for the training process