A scale robust convolutional neural network for automatic building extraction from aerial and satellite imagery

Identifying buildings from remote sensing imagery has been a challenge due to uncertainties from remote sensing imagery and variations in building structure and texture. In this study, we develop a scale robust CNN structure to improve the segmentation accuracy of building data from high-resolution aerial and satellite images. Based on a fully convolutional network, we introduce two Atrous convolutions on the first two lowest-scale layers, respectively, in the decoding step, aiming at enlarging the sight-of-view and integrate semantic information of large buildings. Then, a multi-scale aggregation strategy is applied. The last feature maps of each scale are used to predict the corresponding building labels, and further up-sampled to the original scale and concatenated for the final prediction. In addition, we introduce a combined data augmentation and relative radiometric calibration method for multi-source building extraction. The method enlarges sample spaces and hence the generalization ability of the deep learning models. We validate our developed methods with an aerial dataset of more than 180, 000 buildings with various architectural types, and a satellite image dataset consists of more than 29,000 buildings. The results are compared with several most recent studies. The comparison result shows our neural network outperformed other studies, especially in segmenting scenes of large buildings. The test on transfer learning from aerial dataset to satellite dataset showed our augmentation strategy significantly improved the prediction accuracy; however, further studies are needed to improve the generalization ability of the CNN model