Leverage Spatial Priors and Copy-Paste to Boost Contrastive Learning for Urban-Scene Segmentation

Self-supervised contrastive learning has achieved remarkable performance in computer vision. Its success relies on certain priors that vary from different tasks and data at hand, e.g, the object-centric prior implied by ImageNet. For segmentation on complex scenes, researchers have introduced salient objects or auxiliary labels as priors to guide the pixel similarity modeling but the field is still relatively unexplored. In this work, we attempt to leverage spatial priorsto boost contrastive learning for segmentation on urban scenes, where self-supervised depth estimation is readily available. We argue that the semantics of a pixel can be determined by its adjacent pixels in 3D space without looking at other irrelevant pixels. Based on this concept, we group the pixels based on 3D adjacency and perform copypaste to build cross-context correspondence. We optimize our model to retrieve this challenging correspondence in a contrastive learning manner. Experiments on Cityscapes and KITTI segmentationshow that our method is competitive with other state-of-theart self-supervised representation learning methods, even though our model is only pre-trained with a single GPU on relatively small datasets. Besides, it significantly surpasses the previous baseline by +7.38 mIoU in an unsupervised semantic segmentation setting