If your data distribution shifts, use self-learning
This provides a simple and effective solution for practitioners deploying machine learning models in real-world scenarios with data distribution shifts, though it is incremental as it builds on existing self-learning methods.
The paper tackles the problem of improving computer vision model performance under systematic domain shifts by using self-learning techniques like entropy minimization and pseudo-labeling, achieving state-of-the-art results such as 8.5% error on CIFAR10-C and 22.0% mCE on ImageNet-C.
We demonstrate that self-learning techniques like entropy minimization and pseudo-labeling are simple and effective at improving performance of a deployed computer vision model under systematic domain shifts. We conduct a wide range of large-scale experiments and show consistent improvements irrespective of the model architecture, the pre-training technique or the type of distribution shift. At the same time, self-learning is simple to use in practice because it does not require knowledge or access to the original training data or scheme, is robust to hyperparameter choices, is straight-forward to implement and requires only a few adaptation epochs. This makes self-learning techniques highly attractive for any practitioner who applies machine learning algorithms in the real world. We present state-of-the-art adaptation results on CIFAR10-C (8.5% error), ImageNet-C (22.0% mCE), ImageNet-R (17.4% error) and ImageNet-A (14.8% error), theoretically study the dynamics of self-supervised adaptation methods and propose a new classification dataset (ImageNet-D) which is challenging even with adaptation.