Coarse-Fine Spectral-Aware Deformable Convolution For Hyperspectral Image Reconstruction
This work improves hyperspectral image reconstruction for applications like remote sensing or medical imaging, but it appears incremental as it adapts existing deformable convolution techniques to a specific domain.
The paper tackles the problem of reconstructing hyperspectral images from coded aperture snapshot spectral imaging by addressing limitations of CNNs and Transformers, proposing a novel network that achieves state-of-the-art performance on simulated and real datasets.
We study the inverse problem of Coded Aperture Snapshot Spectral Imaging (CASSI), which captures a spatial-spectral data cube using snapshot 2D measurements and uses algorithms to reconstruct 3D hyperspectral images (HSI). However, current methods based on Convolutional Neural Networks (CNNs) struggle to capture long-range dependencies and non-local similarities. The recently popular Transformer-based methods are poorly deployed on downstream tasks due to the high computational cost caused by self-attention. In this paper, we propose Coarse-Fine Spectral-Aware Deformable Convolution Network (CFSDCN), applying deformable convolutional networks (DCN) to this task for the first time. Considering the sparsity of HSI, we design a deformable convolution module that exploits its deformability to capture long-range dependencies and non-local similarities. In addition, we propose a new spectral information interaction module that considers both coarse-grained and fine-grained spectral similarities. Extensive experiments demonstrate that our CFSDCN significantly outperforms previous state-of-the-art (SOTA) methods on both simulated and real HSI datasets.