Systematic comparison of incomplete-supervision approaches for biomedical image classification.

Deep learning based classification of biomedical images requires expensive manual annotation by experts. Incomplete-supervision approaches including active learning, pre-training, and semi-supervised learning have thus been developed to increase classification performance with a limited number of annotated images. In practice, a combination of these approaches is often used to reach the desired performance for biomedical images. Most of these approaches are designed for natural images, which differ fundamentally from biomedical images in terms of color, contrast, image complexity, and class imbalance. In addition, it is not always clear which combination to use in practical cases. We, therefore, analyzed the performance of combining seven active learning, three pre-training, and two semi-supervised methods on four exemplary biomedical image datasets covering various imaging modalities and resolutions. The results showed that the ImageNet (pre-training) in combination with pseudo-labeling (semi-supervised learning) dominates the best performing combinations, while no particular active learning algorithm prevailed. For three out of four datasets, this combination reached over 90% of the fully supervised results by only adding 25% of labeled data. An ablation study also showed that pre-training and semi-supervised learning contributed up to 25% increase in F1-score in each cycle. In contrast, active learning contributed less than 5% increase in each cycle. Based on these results, we suggest employing the correct combination of pre-training and semi-supervised learning can be more efficient than active learning for biomedical image classification with limited annotated images. We believe that our study is an important step towards annotation-efficient model training for biomedical classification challenges