Deep Transfer Network: Unsupervised Domain Adaptation
This work addresses domain adaptation for machine learning applications where labeled data is scarce in target domains, representing an incremental advancement by combining deep learning with distribution matching techniques.
The paper tackles the problem of domain adaptation by proposing Deep Transfer Network (DTN), which uses deep neural networks to match marginal and conditional distributions between source and target datasets, resulting in significant improvements in execution time and classification accuracy over former methods.
Domain adaptation aims at training a classifier in one dataset and applying it to a related but not identical dataset. One successfully used framework of domain adaptation is to learn a transformation to match both the distribution of the features (marginal distribution), and the distribution of the labels given features (conditional distribution). In this paper, we propose a new domain adaptation framework named Deep Transfer Network (DTN), where the highly flexible deep neural networks are used to implement such a distribution matching process. This is achieved by two types of layers in DTN: the shared feature extraction layers which learn a shared feature subspace in which the marginal distributions of the source and the target samples are drawn close, and the discrimination layers which match conditional distributions by classifier transduction. We also show that DTN has a computation complexity linear to the number of training samples, making it suitable to large-scale problems. By combining the best paradigms in both worlds (deep neural networks in recognition, and matching marginal and conditional distributions in domain adaptation), we demonstrate by extensive experiments that DTN improves significantly over former methods in both execution time and classification accuracy.