Domain Adaptation via Feature Refinement
This work addresses domain adaptation for machine learning models to handle distribution shifts without target labels, but it appears incremental as it builds on existing techniques like Batch Normalization adaptation and feature distillation.
The paper tackled the problem of unsupervised domain adaptation under distribution shift by proposing DAFR2, which outperformed prior methods on benchmark datasets like CIFAR10-C and MNIST-C in robustness to corruption.
We propose Domain Adaptation via Feature Refinement (DAFR2), a simple yet effective framework for unsupervised domain adaptation under distribution shift. The proposed method synergistically combines three key components: adaptation of Batch Normalization statistics using unlabeled target data, feature distillation from a source-trained model and hypothesis transfer. By aligning feature distributions at the statistical and representational levels, DAFR2 produces robust and domain-invariant feature spaces that generalize across similar domains without requiring target labels, complex architectures or sophisticated training objectives. Extensive experiments on benchmark datasets, including CIFAR10-C, CIFAR100-C, MNIST-C and PatchCamelyon-C, demonstrate that the proposed algorithm outperforms prior methods in robustness to corruption. Theoretical and empirical analyses further reveal that our method achieves improved feature alignment, increased mutual information between the domains and reduced sensitivity to input perturbations.