Adaptive Batch Normalization Networks for Adversarial Robustness
This addresses the issue of time-consuming adversarial training for practitioners, offering a more efficient defense method.
The paper tackles the problem of adversarial robustness in deep networks without using adversarial training, proposing Adaptive Batch Normalization Networks (ABNN) that align BN statistics from a substitute model to improve robustness against digital and physical attacks on image and video datasets, achieving higher clean data performance and lower training time complexity compared to adversarial training.
Deep networks are vulnerable to adversarial examples. Adversarial Training (AT) has been a standard foundation of modern adversarial defense approaches due to its remarkable effectiveness. However, AT is extremely time-consuming, refraining it from wide deployment in practical applications. In this paper, we aim at a non-AT defense: How to design a defense method that gets rid of AT but is still robust against strong adversarial attacks? To answer this question, we resort to adaptive Batch Normalization (BN), inspired by the recent advances in test-time domain adaptation. We propose a novel defense accordingly, referred to as the Adaptive Batch Normalization Network (ABNN). ABNN employs a pre-trained substitute model to generate clean BN statistics and sends them to the target model. The target model is exclusively trained on clean data and learns to align the substitute model's BN statistics. Experimental results show that ABNN consistently improves adversarial robustness against both digital and physically realizable attacks on both image and video datasets. Furthermore, ABNN can achieve higher clean data performance and significantly lower training time complexity compared to AT-based approaches.