Deep semantic understanding of high resolution remote sensing image

© 2016 IEEE. With the rapid development of remote sensing technology, huge quantities of high resolution remote sensing images are available now. Understanding these images in semantic level is of great significance. Hence, a deep multimodal neural network model for semantic understanding of the high resolution remote sensing images is proposed in this paper, which uses both visual and textual information of the high resolution remote sensing images to generate natural sentences describing the given images. In the proposed model, the convolution neural network is utilized to extract the image feature, which is then combined with the text descriptions of the images by RNN or LSTMs. And in the experiments, two new remote sensing image-captions datasets are built at first. Then different kinds of CNNs with RNN or LSTMs are combined to find which is the best combination for caption generation. The experiments results prove that the proposed method achieves good performances in semantic understanding of high resolution remote sensing images