Learning to Predict Clinical Outcomes from Soft Tissue Sarcoma MRI

Soft Tissue Sarcomas (STS) are among the most dangerous diseases, with a 50% mortality rate in the USA in 2016. Heterogeneous responses to the treatments of the same sub-type of STS as well as intra-tumor heterogeneity make the study of biopsies imprecise. Radiologists make efforts to find non-invasive approaches to gather useful and important information regarding characteristics and behaviors of STS tumors, such as aggressiveness and recurrence. Quantitative image analysis is an approach to integrate information extracted using data science, such as data mining and machine learning with biological an clinical data to assist radiologists in making the best recommendation on clinical trials and the course of treatment. The new methods in “Radiomics extract meaningful features from medical imaging data for diagnostic and prognostic goals. Furthermore, features extracted from Convolutional Neural Networks (CNNs) are demonstrating very powerful and robust performance in computer aided decision systems (CADs). Also, a well-known computer vision approach, Bag of Visual Words, has recently been applied on imaging data for machine learning purposes such as classification of different types of tumors based on their specific behavior and phenotype. These approaches are not fully and widely investigated in STS. This dissertation provides novel versions of image analysis based on Radiomics and Bag of Visual Words integrated with deep features to quantify the heterogeneity of entire STS as well as sub-regions, which have predictive and prognostic imaging features, from single and multi-sequence Magnetic Resonance Imaging (MRI). STS are types of cancer which are rarely touched in term of quantitative cancer analysis versus other type of cancers such as lung, brain and breast cancers. This dissertation does a comprehensive analysis on available data in 2D and multi-slice to predict the behavior of the STS with regard to clinical outcomes such as recurrence or metastasis and amount of tumor necrosis. The experimental results using Radiomics as well as a new ensemble of Bags of Visual Words framework are promising with 91.66% classification accuracy and 0.91 AUC for metastasis, using ensemble of Bags of Visual Words framework integrated with deep features, and 82.44% classification accuracy with 0.63 AUC for necrosis progression, using Radiomics framework, in tests on the available datasets