XGAN: Unsupervised Image-to-Image Translation for Many-to-Many Mappings
This addresses the challenge of many-to-many image translation for tasks like style transfer, though it appears incremental as it builds on existing GAN and domain adaptation methods.
The paper tackles the problem of unsupervised semantic style transfer between two unpaired image collections, introducing XGAN, a dual adversarial autoencoder that learns a shared semantic representation and bidirectional translations, and reports promising qualitative results for face-to-cartoon translation.
Style transfer usually refers to the task of applying color and texture information from a specific style image to a given content image while preserving the structure of the latter. Here we tackle the more generic problem of semantic style transfer: given two unpaired collections of images, we aim to learn a mapping between the corpus-level style of each collection, while preserving semantic content shared across the two domains. We introduce XGAN ("Cross-GAN"), a dual adversarial autoencoder, which captures a shared representation of the common domain semantic content in an unsupervised way, while jointly learning the domain-to-domain image translations in both directions. We exploit ideas from the domain adaptation literature and define a semantic consistency loss which encourages the model to preserve semantics in the learned embedding space. We report promising qualitative results for the task of face-to-cartoon translation. The cartoon dataset, CartoonSet, we collected for this purpose is publicly available at google.github.io/cartoonset/ as a new benchmark for semantic style transfer.