Synthesis of multispectral images to high spatial resolution: A critical review of fusion methods based on remote sensing physics

Abstract—Our framework is the synthesis of multispectral im-ages (MS) at higher spatial resolution, which should be as close as possible to those that would have been acquired by the corre-sponding sensors if they had this high resolution. This synthesis is performed with the help of a high spatial but low spectral resolution image: the panchromatic (Pan) image. The fusion of the Pan and MS images is classically referred as pan-sharpening. A fused product reaches good quality only if the characteristics and differences between input images are taken into account. Dissimilarities existing between these two data sets originate from two causes—different times and different spectral bands of ac-quisition. Remote sensing physics should be carefully considered while designing the fusion process. Because of the complexity of physics and the large number of unknowns, authors are led to make assumptions to drive their development. Weaknesses and strengths of each reported method are raised and confronted to these physical constraints. The conclusion of this critical survey of literature is that the choice in the assumptions for the develop-ment of a method is crucial, with the risk to drastically weaken fusion performance. It is also shown that the Amélioration de la Résolution Spatiale par Injection de Structures concept prevents from introducing spectral distortion into fused products and offers a reliable framework for further developments. Index Terms—Image enhancement, image processing, merging, multiresolution techniques, remote sensing. I