TuiGAN: Learning Versatile Image-to-Image Translation with Two Unpaired Images
This addresses the challenge of image translation in data-scarce scenarios, offering a versatile solution for applications where collecting numerous images is difficult.
The paper tackles the problem of unsupervised image-to-image translation with very limited data, proposing TuiGAN, which achieves translation using only two unpaired images and outperforms baselines on various tasks while matching state-of-the-art models trained with more data.
An unsupervised image-to-image translation (UI2I) task deals with learning a mapping between two domains without paired images. While existing UI2I methods usually require numerous unpaired images from different domains for training, there are many scenarios where training data is quite limited. In this paper, we argue that even if each domain contains a single image, UI2I can still be achieved. To this end, we propose TuiGAN, a generative model that is trained on only two unpaired images and amounts to one-shot unsupervised learning. With TuiGAN, an image is translated in a coarse-to-fine manner where the generated image is gradually refined from global structures to local details. We conduct extensive experiments to verify that our versatile method can outperform strong baselines on a wide variety of UI2I tasks. Moreover, TuiGAN is capable of achieving comparable performance with the state-of-the-art UI2I models trained with sufficient data.