pyroSAR - a Python Framework for Large-Scale SAR Satellite Data Processing

The launch of recent satellite missions, the Sentinel fleet of the European Copernicus programme in particular, has led to a tremendous increase in available earth observation data provided at no cost. The increase in data availability opens new opportunities for analysing data not only in the spatial but also temporal domain by observing time series and thus the possibility to visualise processes on the earth's surface. Although this is not entirely new to optical satellite data, Synthetic Aperture Radar (SAR) data was only infrequently acquired in the past. With the new ESA SAR satellites Sentinel-1A and Sentinel-1B there is now the possibility to observe the earth's surface with a repeat rate of up to six days and a spatial resolution of 10 m independent of atmospheric effects and sun illumination. Together with the increase in data availability comes the challenge of organizing data and preparing it for scientific analysis. While traditional software aimed at analysing single images, the need arises for fully automated organization of large image archives with thousands of images together with a highly capable processing framework to make full use of available hardware resources. The pyroSAR environment aims at providing a complete solution for the organization and processing of SAR satellite data for applications scalable from personal computers to large server infrastructures using only open source tools and libraries. Its purpose is to provide complex functionality for reading various data formats from past and present SAR satellite missions, handling metadata about acquisition characteristics in a database, and providing homogenised user-friendly access to processing utilities in ESA's Sentinel Application Platform (SNAP) as well as GAMMA Remote Sensing software. The individual processing steps are recorded in homogenized XML workflows so that a dataset is always annotated with information about its way from raw source to analysis ready format. The data reader uses the Geo Data Abstraction Library (GDAL) where possible and own implementations otherwise. The metadata attributes are homogenised to enable database access of specific acquisition characteristics across different sensor platforms. The metadata is ingested into a database from which the original imagery can be selected for processing whilst keeping track of what had been processed before. The SAR processor provides functionality to distribute the tasks across different computing cores as well as different server nodes. By following a stringent naming scheme for processed images as well as annotated metadata XMLs, processing can be organized to be performed by several operators in a server network. This way, redundant usage of disk space and processing resources can be reduced. Once the images are processed, further functionalities are available for mosaicking and resampling images to common pixel boundaries suited for time series analysis or directly exporting the imagery to an Open Data Cube. The overall aim is thus to reduce the entire process of image processing to very few commands in order to achieve maximum usability. By simply providing a test site geometry, the type of data required and some additional parameters, like spatial resolution and time frame, all available raw SAR-data will be retrieved, pre-processed and exported to a selected format. Thus, the time needed for handling data and software, which could otherwise be spent directly on the analysis of data and development of algorithms to derive information from the data, is greatly reduced