Cross-Attention in Coupled Unmixing Nets for Unsupervised Hyperspectral Super-Resolution
This addresses the problem of improving hyperspectral image quality for remote sensing applications, but it is incremental as it builds on existing deep learning and unmixing techniques.
The paper tackles unsupervised hyperspectral image super-resolution by proposing CUCaNet, a coupled unmixing network with cross-attention, which enhances spatial resolution using multispectral images and demonstrates superiority over state-of-the-art models in experiments on three datasets.
The recent advancement of deep learning techniques has made great progress on hyperspectral image super-resolution (HSI-SR). Yet the development of unsupervised deep networks remains challenging for this task. To this end, we propose a novel coupled unmixing network with a cross-attention mechanism, CUCaNet for short, to enhance the spatial resolution of HSI by means of higher-spatial-resolution multispectral image (MSI). Inspired by coupled spectral unmixing, a two-stream convolutional autoencoder framework is taken as backbone to jointly decompose MS and HS data into a spectrally meaningful basis and corresponding coefficients. CUCaNet is capable of adaptively learning spectral and spatial response functions from HS-MS correspondences by enforcing reasonable consistency assumptions on the networks. Moreover, a cross-attention module is devised to yield more effective spatial-spectral information transfer in networks. Extensive experiments are conducted on three widely-used HS-MS datasets in comparison with state-of-the-art HSI-SR models, demonstrating the superiority of the CUCaNet in the HSI-SR application. Furthermore, the codes and datasets will be available at: https://github.com/danfenghong/ECCV2020_CUCaNet.