Revisiting Batch Normalization for Improving Corruption Robustness
This addresses robustness issues for deep learning models in real-world applications where test data may be corrupted, offering a simple, incremental improvement.
The paper tackles the problem of deep neural networks' performance degradation on corrupted images by interpreting corruption as a domain shift and adapting batch normalization statistics without retraining. It improves top1 accuracy on ImageNet-C from 39.2% to 48.7% for ResNet50 and boosts state-of-the-art robust models from 58.1% to 63.3%.
The performance of DNNs trained on clean images has been shown to decrease when the test images have common corruptions. In this work, we interpret corruption robustness as a domain shift and propose to rectify batch normalization (BN) statistics for improving model robustness. This is motivated by perceiving the shift from the clean domain to the corruption domain as a style shift that is represented by the BN statistics. We find that simply estimating and adapting the BN statistics on a few (32 for instance) representation samples, without retraining the model, improves the corruption robustness by a large margin on several benchmark datasets with a wide range of model architectures. For example, on ImageNet-C, statistics adaptation improves the top1 accuracy of ResNet50 from 39.2% to 48.7%. Moreover, we find that this technique can further improve state-of-the-art robust models from 58.1% to 63.3%.