LANIT: Language-Driven Image-to-Image Translation for Unlabeled Data
This work addresses image-to-image translation for unlabeled data, offering a more flexible approach for real-world applications with multiple attributes, though it is incremental in building on existing unsupervised methods.
The paper tackles the problems of reliance on per-sample domain annotations and inability to handle multiple attributes in image-to-image translation by introducing LANIT, a language-driven model that uses text attributes for domain labeling, enabling multi-hot labels and achieving comparable or superior performance on standard benchmarks.
Existing techniques for image-to-image translation commonly have suffered from two critical problems: heavy reliance on per-sample domain annotation and/or inability of handling multiple attributes per image. Recent truly-unsupervised methods adopt clustering approaches to easily provide per-sample one-hot domain labels. However, they cannot account for the real-world setting: one sample may have multiple attributes. In addition, the semantics of the clusters are not easily coupled to the human understanding. To overcome these, we present a LANguage-driven Image-to-image Translation model, dubbed LANIT. We leverage easy-to-obtain candidate attributes given in texts for a dataset: the similarity between images and attributes indicates per-sample domain labels. This formulation naturally enables multi-hot label so that users can specify the target domain with a set of attributes in language. To account for the case that the initial prompts are inaccurate, we also present prompt learning. We further present domain regularization loss that enforces translated images be mapped to the corresponding domain. Experiments on several standard benchmarks demonstrate that LANIT achieves comparable or superior performance to existing models.