Multi-scale task multiple Instance learning for the classification of digital pathology images with global annotations

Whole slide images (WSIs) are high-resolution digitized images of tissue samples, stored including di_erent magni_cation levels. WSIs datasets often include only global annotations, available thanks to pathology reports. Global annotations refer to global _ndings in the high-resolution image and do not include information about the location of the regions of interest or the magni_cation levels used to identify a _nding. This fact can limit the training of machine learning models, as WSIs are usually very large and each magni_cation level includes di_erent information about the tissue. This paper presents a Multi-Scale Task Multiple Instance Learning (MuSTMIL) method, allowing to better exploit data paired with global labels and to combine contextual and detailed information identi_ed at several magni_cation levels. The method is based on a multiple instance learning framework and on a multi-task network, that combines features from several magni_cation levels and produces multiple predictions (a global one and one for each magni_cation level involved). MuSTMIL is evaluated on colon cancer images, on binary and multilabel classi_cation. MuSTMIL shows an improvement in performance in comparison to both single scale and another multi-scale multiple instance learning algorithm, demonstrating that MuSTMIL can help to better deal with global labels targeting full and multi-scale images. Keywords: Multi-Scale Multiple Instance Learning, Multiple Instance Learning, Multiscale approach, Computational pathology