Adversarial Vertex Mixup: Toward Better Adversarially Robust Generalization
This addresses the issue of adversarial robustness for neural network users, offering an incremental improvement over existing adversarial training methods.
The paper tackles the problem of poor adversarially robust generalization in neural networks by identifying Adversarial Feature Overfitting (AFO) and proposing Adversarial Vertex mixup (AVmixup), which significantly improves robust generalization performance and reduces the trade-off between standard accuracy and adversarial robustness, as shown in experiments on datasets like CIFAR10 and CIFAR100.
Adversarial examples cause neural networks to produce incorrect outputs with high confidence. Although adversarial training is one of the most effective forms of defense against adversarial examples, unfortunately, a large gap exists between test accuracy and training accuracy in adversarial training. In this paper, we identify Adversarial Feature Overfitting (AFO), which may cause poor adversarially robust generalization, and we show that adversarial training can overshoot the optimal point in terms of robust generalization, leading to AFO in our simple Gaussian model. Considering these theoretical results, we present soft labeling as a solution to the AFO problem. Furthermore, we propose Adversarial Vertex mixup (AVmixup), a soft-labeled data augmentation approach for improving adversarially robust generalization. We complement our theoretical analysis with experiments on CIFAR10, CIFAR100, SVHN, and Tiny ImageNet, and show that AVmixup significantly improves the robust generalization performance and that it reduces the trade-off between standard accuracy and adversarial robustness.