Unregistered Spectral Image Fusion: Unmixing, Adversarial Learning, and Recoverability
This work addresses the challenging problem of unregistered image fusion for remote sensing applications, offering a novel unsupervised approach with theoretical insights, though it is incremental in advancing existing fusion methods.
The paper tackles the fusion of unregistered hyperspectral and multispectral images to enhance both spatial and spectral resolution, achieving this through an unsupervised framework that integrates spectral unmixing and adversarial learning, with theoretical recoverability guarantees and validation on diverse datasets.
This paper addresses the fusion of a pair of spatially unregistered hyperspectral image (HSI) and multispectral image (MSI) covering roughly overlapping regions. HSIs offer high spectral but low spatial resolution, while MSIs provide the opposite. The goal is to integrate their complementary information to enhance both HSI spatial resolution and MSI spectral resolution. While hyperspectral-multispectral fusion (HMF) has been widely studied, the unregistered setting remains challenging. Many existing methods focus solely on MSI super-resolution, leaving HSI unchanged. Supervised deep learning approaches were proposed for HSI super-resolution, but rely on accurate training data, which is often unavailable. Moreover, theoretical analyses largely address the co-registered case, leaving unregistered HMF poorly understood. In this work, an unsupervised framework is proposed to simultaneously super-resolve both MSI and HSI. The method integrates coupled spectral unmixing for MSI super-resolution with latent-space adversarial learning for HSI super-resolution. Theoretical guarantees on the recoverability of the super-resolution MSI and HSI are established under reasonable generative models -- providing, to our best knowledge, the first such insights for unregistered HMF. The approach is validated on semi-real and real HSI-MSI pairs across diverse conditions.