PGMAN: An Unsupervised Generative Multi-adversarial Network for Pan-sharpening
This work is significant for remote sensing image analysis, as it offers a method to generate higher quality pan-sharpened images without relying on down-sampled data for training, which is a common limitation in existing methods.
The paper addresses the pan-sharpening problem, which involves fusing low-resolution multi-spectral and high-resolution panchromatic images to create a high-resolution multi-spectral image. The authors propose an unsupervised generative multi-adversarial network, PGMAN, which learns directly from full-resolution images, outperforming state-of-the-art methods on GaoFen-2 and QuickBird images.
Pan-sharpening aims at fusing a low-resolution (LR) multi-spectral (MS) image and a high-resolution (HR) panchromatic (PAN) image acquired by a satellite to generate an HR MS image. Many deep learning based methods have been developed in the past few years. However, since there are no intended HR MS images as references for learning, almost all of the existing methods down-sample the MS and PAN images and regard the original MS images as targets to form a supervised setting for training. These methods may perform well on the down-scaled images, however, they generalize poorly to the full-resolution images. To conquer this problem, we design an unsupervised framework that is able to learn directly from the full-resolution images without any preprocessing. The model is built based on a novel generative multi-adversarial network. We use a two-stream generator to extract the modality-specific features from the PAN and MS images, respectively, and develop a dual-discriminator to preserve the spectral and spatial information of the inputs when performing fusion. Furthermore, a novel loss function is introduced to facilitate training under the unsupervised setting. Experiments and comparisons with other state-of-the-art methods on GaoFen-2 and QuickBird images demonstrate that the proposed method can obtain much better fusion results on the full-resolution images.