The challenge of creating the SARptical dataset

The SARptical dataset consists of about 10,000 matching pairs of very high resolution optical and SAR image patches extracted from TerraSAR-X very high-resolution spotlight images and aerial UltraCAM optical images in dense urban areas of the city of Berlin, Germany. This dataset is distinct from any other existing SAR optical dataset, because the 3D location of the center pixels of the SAR and optical were explicitly matched. Still, creating such dataset poses a fundamental challenge. The reason is that a pixel level matching between SAR and optical images is generally impossible to achieve a without the assistance of a precise 3-D model. This is especially true in dense urban areas, because of the inevitable layover caused by the side-looking SAR imaging geometry. Such misalignment will affect applications like joint classification using SAR and optical images, and raise the difficulty in designing machine learning algorithms. This paper will discuss the challenge of jointly using SAR and optical images for remote sensing applications and propose possible methods to mitigate those misalignment errors