WAT: Improve the Worst-class Robustness in Adversarial Training
This addresses the issue of inconsistent robust accuracy across classes in adversarial training for deep neural networks, which is an incremental improvement over prior methods.
The paper tackles the problem of robustness disparity among classes in adversarial training, proposing a worst-class adversarial training framework that improves worst-class robust accuracy while sacrificing minimal average robust accuracy, with experiments showing it outperforms state-of-the-art methods.
Deep Neural Networks (DNN) have been shown to be vulnerable to adversarial examples. Adversarial training (AT) is a popular and effective strategy to defend against adversarial attacks. Recent works (Benz et al., 2020; Xu et al., 2021; Tian et al., 2021) have shown that a robust model well-trained by AT exhibits a remarkable robustness disparity among classes, and propose various methods to obtain consistent robust accuracy across classes. Unfortunately, these methods sacrifice a good deal of the average robust accuracy. Accordingly, this paper proposes a novel framework of worst-class adversarial training and leverages no-regret dynamics to solve this problem. Our goal is to obtain a classifier with great performance on worst-class and sacrifice just a little average robust accuracy at the same time. We then rigorously analyze the theoretical properties of our proposed algorithm, and the generalization error bound in terms of the worst-class robust risk. Furthermore, we propose a measurement to evaluate the proposed method in terms of both the average and worst-class accuracies. Experiments on various datasets and networks show that our proposed method outperforms the state-of-the-art approaches.