Bridging the Performance Gap between FGSM and PGD Adversarial Training
This work addresses the problem of slow training times for robust deep learning models, offering a more efficient alternative for researchers and practitioners in adversarial machine learning, though it is incremental in improving existing methods.
The paper tackled the trade-off between training efficiency and adversarial robustness in adversarial training by extending the fast gradient sign method (adv.FGSM) with curvature regularization (adv.FGSMR) to match the robustness of the projected gradient descent method (adv.PGD). The result showed that adv.FGSMR achieved comparable robustness on MNIST and better performance on CIFAR-10 under white-box attacks, while being more efficient than adv.PGD.
Deep learning achieves state-of-the-art performance in many tasks but exposes to the underlying vulnerability against adversarial examples. Across existing defense techniques, adversarial training with the projected gradient decent attack (adv.PGD) is considered as one of the most effective ways to achieve moderate adversarial robustness. However, adv.PGD requires too much training time since the projected gradient attack (PGD) takes multiple iterations to generate perturbations. On the other hand, adversarial training with the fast gradient sign method (adv.FGSM) takes much less training time since the fast gradient sign method (FGSM) takes one step to generate perturbations but fails to increase adversarial robustness. In this work, we extend adv.FGSM to make it achieve the adversarial robustness of adv.PGD. We demonstrate that the large curvature along FGSM perturbed direction leads to a large difference in performance of adversarial robustness between adv.FGSM and adv.PGD, and therefore propose combining adv.FGSM with a curvature regularization (adv.FGSMR) in order to bridge the performance gap between adv.FGSM and adv.PGD. The experiments show that adv.FGSMR has higher training efficiency than adv.PGD. In addition, it achieves comparable performance of adversarial robustness on MNIST dataset under white-box attack, and it achieves better performance than adv.PGD under white-box attack and effectively defends the transferable adversarial attack on CIFAR-10 dataset.