Deep Learning for Overhead Imagery: Algorithms and Applications

Remote sensing using overhead imagery has critical impact to the way we understand our environment and offers crucial information for scene understanding, climate change research, disaster response, urban planning, forest management, and many other applications. At present, deep learning is increasingly used in remote sensing, but mostly borrowing algorithms developed for natural images in the computer vision community. Specific challenges arise while applying deep learning to remote sensing. These challenges include issues related to the high dimensionality and limited labeled data, security and robustness to adversarial attacks, and model generalization. In this thesis we focus on tackling these key challenges. We present an end-to-end framework to effectively integrate input feature subset selection into the training procedure of a deep neural network for dimensionality reduction. We show that our framework significantly improves performance on multispectral imagery applications. We evaluate quantitatively the robustness of multispectral and hyperspectral image-based deep learning models to adversarial examples. Our experiments show that methods for generating adversarial examples designed for natural images are also effective for remote sensing imagery. We also introduce a framework that integrates dimensionality reduction, adversarial training, and a detector network that greatly improves models’ robustness without sacrificing performance. We then present a novel network architecture which exploits conditional information to improve generalization of deep learning models. Finally, we propose a new normalization layer which facilitates transfer learning and improves performance across a great variety of tasks. Local context normalization is a very efficient generalization of previous ones, it is invariant to batch size, and it is well-suited for transfer learning and interactive systems.This novel normalization layer provides state-of-the-art performance for the tasks of object detection, semantic segmentation, instance segmentation, and aerial image labeling