Combining Semantic Self-Supervision and Self-Training for Domain Adaptation in Semantic Segmentation

This work presents a two-staged, unsupervised domain adaptation process for semantic segmentation models by combining a self-training and self-supervision strategy. Self-training (i. e., training a model on self-inferred pseudo-labels) yields competitive results for domain adaptation in recent research. However, self-training depends on high-quality pseudo-labels. On the other hand, self-supervision trains the model on a surrogate task and improves its performance on the target domain without further prerequisites. Therefore, our approach improves the model's performance on the target domain with a novel surrogate task. To that, we continuously determine class centroids of the feature representations in the network’s pre-logit layer on the source domain. Our surrogate task clusters the pre-logit feature representations on the target domain regarding these class centroids during both training stages. After the first stage, the resulting model delivers improved pseudo-labels for the additional self-training in the second stage. We evaluate our method on two different domain adaptions, a real-world domain change from Cityscapes to the Berkeley Deep Drive dataset and a synthetic to real-world domain change from GTA5 to the Cityscapes dataset. For the real-world domain change, the evaluation shows a significant improvement of the model from 46% mIoU to 54% mIoU on the target domain. For the synthetic to real-world domain change, we achieve an improvement from 38.8% to 46.42% on the real-world target domai