Integrating Categorical Semantics into Unsupervised Domain Translation
This work addresses a specific bottleneck in domain translation for computer vision applications, offering incremental improvements.
The paper tackled the problem of improving unsupervised domain translation by incorporating categorical semantic features, resulting in better translation between domains with multiple object categories, such as preserving digits in MNIST↔SVHN and enhancing realism in Sketches→Reals.
While unsupervised domain translation (UDT) has seen a lot of success recently, we argue that mediating its translation via categorical semantic features could broaden its applicability. In particular, we demonstrate that categorical semantics improves the translation between perceptually different domains sharing multiple object categories. We propose a method to learn, in an unsupervised manner, categorical semantic features (such as object labels) that are invariant of the source and target domains. We show that conditioning the style encoder of unsupervised domain translation methods on the learned categorical semantics leads to a translation preserving the digits on MNIST$\leftrightarrow$SVHN and to a more realistic stylization on Sketches$\to$Reals.