Multi-Modality Deep Network for JPEG Artifacts Reduction
This addresses the challenge of reconstructing high-quality images from highly compressed data, which is incremental for image processing applications.
The paper tackles the problem of reducing JPEG artifacts in extreme low-bitrate compressed images by proposing a multimodal fusion method that uses text descriptions as prior and supplementary information, achieving better deblocking results than state-of-the-art methods as proven by experiments and a user study.
In recent years, many convolutional neural network-based models are designed for JPEG artifacts reduction, and have achieved notable progress. However, few methods are suitable for extreme low-bitrate image compression artifacts reduction. The main challenge is that the highly compressed image loses too much information, resulting in reconstructing high-quality image difficultly. To address this issue, we propose a multimodal fusion learning method for text-guided JPEG artifacts reduction, in which the corresponding text description not only provides the potential prior information of the highly compressed image, but also serves as supplementary information to assist in image deblocking. We fuse image features and text semantic features from the global and local perspectives respectively, and design a contrastive loss built upon contrastive learning to produce visually pleasing results. Extensive experiments, including a user study, prove that our method can obtain better deblocking results compared to the state-of-the-art methods.