Higher fidelity perceptual image and video compression with a latent conditioned residual denoising diffusion model
This work addresses the fidelity-perception trade-off in image and video compression for applications requiring high-quality reconstructions, representing an incremental improvement over existing diffusion-based methods.
The paper tackles the problem of diffusion-based image compression methods sacrificing fidelity for perceptual quality by proposing a hybrid scheme that combines a decoder network with a latent conditioned diffusion model to refine reconstructions. The approach achieves up to +2dB PSNR improvements while maintaining comparable perceptual scores on standard benchmarks and extends effectively to video compression.
Denoising diffusion models achieved impressive results on several image generation tasks often outperforming GAN based models. Recently, the generative capabilities of diffusion models have been employed for perceptual image compression, such as in CDC. A major drawback of these diffusion-based methods is that, while producing impressive perceptual quality images they are dropping in fidelity/increasing the distortion to the original uncompressed images when compared with other traditional or learned image compression schemes aiming for fidelity. In this paper, we propose a hybrid compression scheme optimized for perceptual quality, extending the approach of the CDC model with a decoder network in order to reduce the impact on distortion metrics such as PSNR. After using the decoder network to generate an initial image, optimized for distortion, the latent conditioned diffusion model refines the reconstruction for perceptual quality by predicting the residual. On standard benchmarks, we achieve up to +2dB PSNR fidelity improvements while maintaining comparable LPIPS and FID perceptual scores when compared with CDC. Additionally, the approach is easily extensible to video compression, where we achieve similar results.