Hidden Covariate Shift: A Minimal Assumption For Domain Adaptation
This addresses domain adaptation for machine learning models when labeled data is available only in a source domain, but it is incremental as it builds on existing assumptions like domain invariant representations.
The paper tackles the problem of unsupervised domain adaptation by proposing the Hidden Covariate Shift hypothesis as a minimal assumption, which involves learning representations where the label distribution conditioned on the representation is domain invariant. It reports state-of-the-art performances on the Amazon Reviews dataset, demonstrating viability.
Unsupervised Domain Adaptation aims to learn a model on a source domain with labeled data in order to perform well on unlabeled data of a target domain. Current approaches focus on learning \textit{Domain Invariant Representations}. It relies on the assumption that such representations are well-suited for learning the supervised task in the target domain. We rather believe that a better and minimal assumption for performing Domain Adaptation is the \textit{Hidden Covariate Shift} hypothesis. Such approach consists in learning a representation of the data such that the label distribution conditioned on this representation is domain invariant. From the Hidden Covariate Shift assumption, we derive an optimization procedure which learns to match an estimated joint distribution on the target domain and a re-weighted joint distribution on the source domain. The re-weighting is done in the representation space and is learned during the optimization procedure. We show on synthetic data and real world data that our approach deals with both \textit{Target Shift} and \textit{Concept Drift}. We report state-of-the-art performances on Amazon Reviews dataset \cite{blitzer2007biographies} demonstrating the viability of this approach.