SALT: Subspace Alignment as an Auxiliary Learning Task for Domain Adaptation
It addresses domain adaptation for machine learning applications where labeled data is scarce, but the approach is incremental as it combines existing geometric and model-based methods.
The paper tackles unsupervised domain adaptation by proposing SALT, a framework that uses subspace alignment as an auxiliary task to improve target performance, achieving results comparable to or better than state-of-the-art on standard benchmarks.
Unsupervised domain adaptation aims to transfer and adapt knowledge learned from a labeled source domain to an unlabeled target domain. Key components of unsupervised domain adaptation include: (a) maximizing performance on the target, and (b) aligning the source and target domains. Traditionally, these tasks have either been considered as separate, or assumed to be implicitly addressed together with high-capacity feature extractors. When considered separately, alignment is usually viewed as a problem of aligning data distributions, either through geometric approaches such as subspace alignment or through distributional alignment such as optimal transport. This paper represents a hybrid approach, where we assume simplified data geometry in the form of subspaces, and consider alignment as an auxiliary task to the primary task of maximizing performance on the source. The alignment is made rather simple by leveraging tractable data geometry in the form of subspaces. We synergistically allow certain parameters derived from the closed-form auxiliary solution, to be affected by gradients from the primary task. The proposed approach represents a unique fusion of geometric and model-based alignment with gradients from a data-driven primary task. Our approach termed SALT, is a simple framework that achieves comparable or sometimes outperforms state-of-the-art on multiple standard benchmarks.