DMDC: Dynamic-mask-based dual camera design for snapshot Hyperspectral Imaging
This work addresses performance limitations in hyperspectral imaging for applications like remote sensing or medical imaging, offering a significant but incremental advance by integrating dynamic masks and dual inputs.
The paper tackles the slow improvement in reconstruction accuracy of coded aperture snapshot spectral imaging (CASSI) by introducing a dynamic-mask-based dual camera system that uses RGB images to guide mask encoding and combines inputs for reconstruction, achieving over 9 dB PSNR improvement over state-of-the-art methods.
Deep learning methods are developing rapidly in coded aperture snapshot spectral imaging (CASSI). The number of parameters and FLOPs of existing state-of-the-art methods (SOTA) continues to increase, but the reconstruction accuracy improves slowly. Current methods still face two problems: 1) The performance of the spatial light modulator (SLM) is not fully developed due to the limitation of fixed Mask coding. 2) The single input limits the network performance. In this paper we present a dynamic-mask-based dual camera system, which consists of an RGB camera and a CASSI system running in parallel. First, the system learns the spatial feature distribution of the scene based on the RGB images, then instructs the SLM to encode each scene, and finally sends both RGB and CASSI images to the network for reconstruction. We further designed the DMDC-net, which consists of two separate networks, a small-scale CNN-based dynamic mask network for dynamic adjustment of the mask and a multimodal reconstruction network for reconstruction using RGB and CASSI measurements. Extensive experiments on multiple datasets show that our method achieves more than 9 dB improvement in PSNR over the SOTA. (https://github.com/caizeyu1992/DMDC)