Exploring Patch-wise Semantic Relation for Contrastive Learning in Image-to-Image Translation Tasks
This addresses a limitation in image-to-image translation for computer vision applications, though it appears incremental as it builds on existing contrastive learning approaches.
The paper tackles the problem of contrastive learning-based image translation methods ignoring diverse semantic relations within images by proposing a semantic relation consistency regularization with decoupled contrastive learning and hard negative mining. The method achieved state-of-the-art performance in single-modal and multi-modal image translations and GAN compression tasks.
Recently, contrastive learning-based image translation methods have been proposed, which contrasts different spatial locations to enhance the spatial correspondence. However, the methods often ignore the diverse semantic relation within the images. To address this, here we propose a novel semantic relation consistency (SRC) regularization along with the decoupled contrastive learning, which utilize the diverse semantics by focusing on the heterogeneous semantics between the image patches of a single image. To further improve the performance, we present a hard negative mining by exploiting the semantic relation. We verified our method for three tasks: single-modal and multi-modal image translations, and GAN compression task for image translation. Experimental results confirmed the state-of-art performance of our method in all the three tasks.