Fusformer: A Transformer-based Fusion Approach for Hyperspectral Image Super-resolution
This work addresses the problem of poor spatial resolution in hyperspectral imaging for applications requiring detailed spectral and spatial data, representing an incremental improvement by applying transformers to a known bottleneck in existing CNN-based methods.
The paper tackles hyperspectral image super-resolution by fusing low-resolution hyperspectral and high-resolution multispectral images using a transformer-based network, achieving superior performance over state-of-the-art methods as shown by various experiments and quality indexes.
Hyperspectral image has become increasingly crucial due to its abundant spectral information. However, It has poor spatial resolution with the limitation of the current imaging mechanism. Nowadays, many convolutional neural networks have been proposed for the hyperspectral image super-resolution problem. However, convolutional neural network (CNN) based methods only consider the local information instead of the global one with the limited kernel size of receptive field in the convolution operation. In this paper, we design a network based on the transformer for fusing the low-resolution hyperspectral images and high-resolution multispectral images to obtain the high-resolution hyperspectral images. Thanks to the representing ability of the transformer, our approach is able to explore the intrinsic relationships of features globally. Furthermore, considering the LR-HSIs hold the main spectral structure, the network focuses on the spatial detail estimation releasing from the burden of reconstructing the whole data. It reduces the mapping space of the proposed network, which enhances the final performance. Various experiments and quality indexes show our approach's superiority compared with other state-of-the-art methods.