Information-Theoretic Analysis of Unsupervised Domain Adaptation
This work addresses the challenge of domain adaptation for machine learning practitioners by providing theoretical insights and practical methods, though it is incremental as it builds on existing analysis.
This paper tackles the problem of analyzing generalization error in unsupervised domain adaptation (UDA) by using information-theoretic tools to derive novel upper bounds for two types of errors, and it presents two simple techniques that are validated experimentally to improve generalization.
This paper uses information-theoretic tools to analyze the generalization error in unsupervised domain adaptation (UDA). We present novel upper bounds for two notions of generalization errors. The first notion measures the gap between the population risk in the target domain and that in the source domain, and the second measures the gap between the population risk in the target domain and the empirical risk in the source domain. While our bounds for the first kind of error are in line with the traditional analysis and give similar insights, our bounds on the second kind of error are algorithm-dependent, which also provide insights into algorithm designs. Specifically, we present two simple techniques for improving generalization in UDA and validate them experimentally.