DACESR: Degradation-Aware Conditional Embedding for Real-World Image Super-Resolution
This work addresses image super-resolution for degraded real-world images, offering an incremental improvement by enhancing existing multimodal models with a novel embedding strategy.
The paper tackles real-world image super-resolution by addressing the limited ability of multimodal large models on degraded images, proposing a degradation-aware conditional embedding method that improves recognition performance and helps networks balance fidelity and perceptual quality, with experiments showing significant gains.
Multimodal large models have shown excellent ability in addressing image super-resolution in real-world scenarios by leveraging language class as condition information, yet their abilities in degraded images remain limited. In this paper, we first revisit the capabilities of the Recognize Anything Model (RAM) for degraded images by calculating text similarity. We find that directly using contrastive learning to fine-tune RAM in the degraded space is difficult to achieve acceptable results. To address this issue, we employ a degradation selection strategy to propose a Real Embedding Extractor (REE), which achieves significant recognition performance gain on degraded image content through contrastive learning. Furthermore, we use a Conditional Feature Modulator (CFM) to incorporate the high-level information of REE for a powerful Mamba-based network, which can leverage effective pixel information to restore image textures and produce visually pleasing results. Extensive experiments demonstrate that the REE can effectively help image super-resolution networks balance fidelity and perceptual quality, highlighting the great potential of Mamba in real-world applications. The source code of this work will be made publicly available at: https://github.com/nathan66666/DACESR.git