Ultra Lowrate Image Compression with Semantic Residual Coding and Compression-aware Diffusion
This work addresses the challenge of improving ultra lowrate image compression for applications requiring high efficiency and quality, representing a novel integration rather than a foundational breakthrough.
The paper tackles the problem of suboptimal performance in image compression by proposing ResULIC, a method that integrates semantic residual coding and compression-aware diffusion to enhance reconstruction fidelity and coding efficiency, achieving significant BD-rate savings of -80.7% and -66.3% in LPIPS and FID metrics compared to state-of-the-art methods.
Existing multimodal large model-based image compression frameworks often rely on a fragmented integration of semantic retrieval, latent compression, and generative models, resulting in suboptimal performance in both reconstruction fidelity and coding efficiency. To address these challenges, we propose a residual-guided ultra lowrate image compression named ResULIC, which incorporates residual signals into both semantic retrieval and the diffusion-based generation process. Specifically, we introduce Semantic Residual Coding (SRC) to capture the semantic disparity between the original image and its compressed latent representation. A perceptual fidelity optimizer is further applied for superior reconstruction quality. Additionally, we present the Compression-aware Diffusion Model (CDM), which establishes an optimal alignment between bitrates and diffusion time steps, improving compression-reconstruction synergy. Extensive experiments demonstrate the effectiveness of ResULIC, achieving superior objective and subjective performance compared to state-of-the-art diffusion-based methods with - 80.7%, -66.3% BD-rate saving in terms of LPIPS and FID. Project page is available at https: //njuvision.github.io/ResULIC/.