SpectraMorph: Structured Latent Learning for Self-Supervised Hyperspectral Super-Resolution
This addresses the issue of low spatial resolution in hyperspectral imaging for remote sensing and computer vision applications, offering an incremental improvement with enhanced interpretability and efficiency.
The paper tackles the problem of hyperspectral super-resolution by fusing low-resolution hyperspectral images with high-resolution multispectral images, proposing SpectraMorph, a physics-guided self-supervised framework that uses an unmixing bottleneck for interpretability and robustness, achieving state-of-the-art performance in unsupervised/self-supervised settings and competitive results against supervised methods.
Hyperspectral sensors capture dense spectra per pixel but suffer from low spatial resolution, causing blurred boundaries and mixed-pixel effects. Co-registered companion sensors such as multispectral, RGB, or panchromatic cameras provide high-resolution spatial detail, motivating hyperspectral super-resolution through the fusion of hyperspectral and multispectral images (HSI-MSI). Existing deep learning based methods achieve strong performance but rely on opaque regressors that lack interpretability and often fail when the MSI has very few bands. We propose SpectraMorph, a physics-guided self-supervised fusion framework with a structured latent space. Instead of direct regression, SpectraMorph enforces an unmixing bottleneck: endmember signatures are extracted from the low-resolution HSI, and a compact multilayer perceptron predicts abundance-like maps from the MSI. Spectra are reconstructed by linear mixing, with training performed in a self-supervised manner via the MSI sensor's spectral response function. SpectraMorph produces interpretable intermediates, trains in under a minute, and remains robust even with a single-band (pan-chromatic) MSI. Experiments on synthetic and real-world datasets show SpectraMorph consistently outperforming state-of-the-art unsupervised/self-supervised baselines while remaining very competitive against supervised baselines.