Beyond Separability: Analyzing the Linear Transferability of Contrastive Representations to Related Subpopulations
This provides theoretical insights for unsupervised domain adaptation, though it is incremental as it builds on existing contrastive learning frameworks.
The paper analyzes when contrastive representations exhibit linear transferability across domains, proving it occurs when same-class data across domains are more related than different-class data, even with unbounded density ratios and distant representations.
Contrastive learning is a highly effective method for learning representations from unlabeled data. Recent works show that contrastive representations can transfer across domains, leading to simple state-of-the-art algorithms for unsupervised domain adaptation. In particular, a linear classifier trained to separate the representations on the source domain can also predict classes on the target domain accurately, even though the representations of the two domains are far from each other. We refer to this phenomenon as linear transferability. This paper analyzes when and why contrastive representations exhibit linear transferability in a general unsupervised domain adaptation setting. We prove that linear transferability can occur when data from the same class in different domains (e.g., photo dogs and cartoon dogs) are more related with each other than data from different classes in different domains (e.g., photo dogs and cartoon cats) are. Our analyses are in a realistic regime where the source and target domains can have unbounded density ratios and be weakly related, and they have distant representations across domains.