Multi-task Deep Segmentation and Radiomics for Automatic Prognosis in Head and Neck Cancer

International audienceWe propose a novel method for the prediction of patient prognosis with Head and Neck cancer (HandN) from FDG-PET/CT images. In particular, we aim at automatically predicting Disease-Free Survival (DFS) for patients treated with radiotherapy or both radiotherapy and chemotherapy. We design a multi-task deep UNet to learn both the segmentation of the primary Gross Tumor Volume (GTVt) and the outcome of the patient from PET and CT images. The motivation for this approach lies in the complementarity of the two tasks and the shared visual features relevant to both tasks. A multi-modal (PET and CT) 3D UNet is trained with a combination of survival and Dice losses to jointly learn the two tasks. The model is evaluated on the HECKTOR 2020 dataset consisting of 239 HandN patients with PET, CT, GTVt contours and DFS data (five centers). The results are compared with a standard Cox PET/CT radiomics model. The proposed multi-task CNN reaches a C-index of 0.723, outperforming both the deep radiomics model without segmentation (C-index of 0.650) and the standard radiomics model (C-index of 0.695). Besides the improved performance in outcome prediction, the main advantage of the proposed multi-task approach is that it can predict patient prognosis without a manual delineation of the GTVt, a tedious and time-consuming process that hinders the validation of large-scale radiomics studies. The code will be shared for reproducibility on our GitHub repository. © 2021, Springer Nature Switzerland AG