Towards the Desirable Decision Boundary by Moderate-Margin Adversarial Training
This work addresses the robustness-accuracy trade-off in adversarial defense for deep learning models, presenting an incremental improvement over existing adversarial training methods.
The paper tackles the trade-off between robustness and natural accuracy in adversarial training by proposing Moderate-Margin Adversarial Training (MMAT), which learns a moderate-inclusive decision boundary to reduce cross-over between natural and adversarial examples, achieving state-of-the-art robustness and natural accuracy on SVHN.
Adversarial training, as one of the most effective defense methods against adversarial attacks, tends to learn an inclusive decision boundary to increase the robustness of deep learning models. However, due to the large and unnecessary increase in the margin along adversarial directions, adversarial training causes heavy cross-over between natural examples and adversarial examples, which is not conducive to balancing the trade-off between robustness and natural accuracy. In this paper, we propose a novel adversarial training scheme to achieve a better trade-off between robustness and natural accuracy. It aims to learn a moderate-inclusive decision boundary, which means that the margins of natural examples under the decision boundary are moderate. We call this scheme Moderate-Margin Adversarial Training (MMAT), which generates finer-grained adversarial examples to mitigate the cross-over problem. We also take advantage of logits from a teacher model that has been well-trained to guide the learning of our model. Finally, MMAT achieves high natural accuracy and robustness under both black-box and white-box attacks. On SVHN, for example, state-of-the-art robustness and natural accuracy are achieved.