Bayesian Uncertainty Matching for Unsupervised Domain Adaptation
This work addresses domain shift issues in machine learning for applications like computer vision, offering a novel approach to improve model robustness when training and test data differ, though it is incremental as it builds on existing distribution matching techniques.
The paper tackled performance degradation in unsupervised domain adaptation caused by label distribution shift at the classifier, proposing a method that matches joint distributions using Bayesian uncertainty to achieve consistent predictions across domains, with results showing superiority over state-of-the-art methods on three benchmark datasets, especially in alleviating negative transfer.
Domain adaptation is an important technique to alleviate performance degradation caused by domain shift, e.g., when training and test data come from different domains. Most existing deep adaptation methods focus on reducing domain shift by matching marginal feature distributions through deep transformations on the input features, due to the unavailability of target domain labels. We show that domain shift may still exist via label distribution shift at the classifier, thus deteriorating model performances. To alleviate this issue, we propose an approximate joint distribution matching scheme by exploiting prediction uncertainty. Specifically, we use a Bayesian neural network to quantify prediction uncertainty of a classifier. By imposing distribution matching on both features and labels (via uncertainty), label distribution mismatching in source and target data is effectively alleviated, encouraging the classifier to produce consistent predictions across domains. We also propose a few techniques to improve our method by adaptively reweighting domain adaptation loss to achieve nontrivial distribution matching and stable training. Comparisons with state of the art unsupervised domain adaptation methods on three popular benchmark datasets demonstrate the superiority of our approach, especially on the effectiveness of alleviating negative transfer.