Unbalanced Feature Transport for Exemplar-based Image Translation
This work addresses the problem of realistic image generation with reference styles for researchers and practitioners in computer vision, representing an incremental improvement over existing GAN-based methods.
The paper tackles the challenge of generating high-fidelity images in conditional image-to-image translation by introducing a framework that uses unbalanced optimal transport for feature alignment between inputs and style exemplars, achieving superior results in multiple tasks compared to state-of-the-art methods.
Despite the great success of GANs in images translation with different conditioned inputs such as semantic segmentation and edge maps, generating high-fidelity realistic images with reference styles remains a grand challenge in conditional image-to-image translation. This paper presents a general image translation framework that incorporates optimal transport for feature alignment between conditional inputs and style exemplars in image translation. The introduction of optimal transport mitigates the constraint of many-to-one feature matching significantly while building up accurate semantic correspondences between conditional inputs and exemplars. We design a novel unbalanced optimal transport to address the transport between features with deviational distributions which exists widely between conditional inputs and exemplars. In addition, we design a semantic-activation normalization scheme that injects style features of exemplars into the image translation process successfully. Extensive experiments over multiple image translation tasks show that our method achieves superior image translation qualitatively and quantitatively as compared with the state-of-the-art.