MP-HSIR: A Multi-Prompt Framework for Universal Hyperspectral Image Restoration
This work addresses the limitation of existing methods that rely on specific degradation assumptions, offering a universal solution for hyperspectral image restoration in complex scenarios, though it appears incremental as it builds on transformer and prompt-based techniques.
The paper tackles the problem of hyperspectral image restoration under diverse and unknown degradations by proposing MP-HSIR, a multi-prompt framework that integrates spectral, textual, and visual prompts, achieving state-of-the-art performance across 9 restoration tasks and outperforming both all-in-one and task-specific methods.
Hyperspectral images (HSIs) often suffer from diverse and unknown degradations during imaging, leading to severe spectral and spatial distortions. Existing HSI restoration methods typically rely on specific degradation assumptions, limiting their effectiveness in complex scenarios. In this paper, we propose \textbf{MP-HSIR}, a novel multi-prompt framework that effectively integrates spectral, textual, and visual prompts to achieve universal HSI restoration across diverse degradation types and intensities. Specifically, we develop a prompt-guided spatial-spectral transformer, which incorporates spatial self-attention and a prompt-guided dual-branch spectral self-attention. Since degradations affect spectral features differently, we introduce spectral prompts in the local spectral branch to provide universal low-rank spectral patterns as prior knowledge for enhancing spectral reconstruction. Furthermore, the text-visual synergistic prompt fuses high-level semantic representations with fine-grained visual features to encode degradation information, thereby guiding the restoration process. Extensive experiments on 9 HSI restoration tasks, including all-in-one scenarios, generalization tests, and real-world cases, demonstrate that MP-HSIR not only consistently outperforms existing all-in-one methods but also surpasses state-of-the-art task-specific approaches across multiple tasks. The code and models are available at https://github.com/ZhehuiWu/MP-HSIR.