Risk Bounds for Unsupervised Cross-Domain Mapping with IPMs
This work addresses a theoretical gap for researchers and practitioners using cross-domain mapping, though it is incremental as it builds on existing adversarial methods.
The paper tackles the lack of theoretical guarantees for unsupervised cross-domain mapping algorithms by deriving a novel risk bound based on IPMs, which upper bounds the risk between learned and target mappings and leads to algorithmic improvements like hyperparameter selection and confidence estimation.
The recent empirical success of unsupervised cross-domain mapping algorithms, between two domains that share common characteristics, is not well-supported by theoretical justifications. This lacuna is especially troubling, given the clear ambiguity in such mappings. We work with adversarial training methods based on IPMs and derive a novel risk bound, which upper bounds the risk between the learned mapping $h$ and the target mapping $y$, by a sum of three terms: (i) the risk between $h$ and the most distant alternative mapping that was learned by the same cross-domain mapping algorithm, (ii) the minimal discrepancy between the target domain and the domain obtained by applying a hypothesis $h^*$ on the samples of the source domain, where $h^*$ is a hypothesis selectable by the same algorithm. The bound is directly related to Occam's razor and encourages the selection of the minimal architecture that supports a small mapping discrepancy and (iii) an approximation error term that decreases as the complexity of the class of discriminators increases and is empirically shown to be small. The bound leads to multiple algorithmic consequences, including a method for hyperparameters selection and for early stopping in cross-domain mapping GANs. We also demonstrate a novel capability for unsupervised learning of estimating confidence in the mapping of every specific sample.