Towards Accurate Model Selection in Deep Unsupervised Domain Adaptation
Addresses the model selection bottleneck for practitioners comparing Deep UDA methods, offering a principled solution to a known problem.
Deep UDA lacks accurate model selection methods; the proposed DEV provides unbiased target risk estimation with bounded variance, reducing variance via control variates, and is validated theoretically and empirically.
Deep unsupervised domain adaptation (Deep UDA) methods successfully leverage rich labeled data in a source domain to boost the performance on related but unlabeled data in a target domain. However, algorithm comparison is cumbersome in Deep UDA due to the absence of accurate and standardized model selection method, posing an obstacle to further advances in the field. Existing model selection methods for Deep UDA are either highly biased, restricted, unstable, or even controversial (requiring labeled target data). To this end, we propose \textit{Deep Embedded Validation} (\textbf{DEV}), which embeds adapted feature representation into the validation procedure to obtain unbiased estimation of the target risk with bounded variance. The variance is further reduced by the technique of control variate. The efficacy of the method has been justified both theoretically and empirically.