Adversarial Training on Purification (AToP): Advancing Both Robustness and Generalization
This addresses the problem of adversarial vulnerability in deep neural networks for security-critical applications, offering a hybrid approach that is incremental but aims to balance robustness and generalization.
The paper tackles the trade-off between robustness and generalization in adversarial defense, proposing AToP to achieve optimal robustness against known attacks while generalizing to unseen ones, with experiments on CIFAR-10, CIFAR-100, and ImageNette showing improved performance.
The deep neural networks are known to be vulnerable to well-designed adversarial attacks. The most successful defense technique based on adversarial training (AT) can achieve optimal robustness against particular attacks but cannot generalize well to unseen attacks. Another effective defense technique based on adversarial purification (AP) can enhance generalization but cannot achieve optimal robustness. Meanwhile, both methods share one common limitation on the degraded standard accuracy. To mitigate these issues, we propose a novel pipeline to acquire the robust purifier model, named Adversarial Training on Purification (AToP), which comprises two components: perturbation destruction by random transforms (RT) and purifier model fine-tuned (FT) by adversarial loss. RT is essential to avoid overlearning to known attacks, resulting in the robustness generalization to unseen attacks, and FT is essential for the improvement of robustness. To evaluate our method in an efficient and scalable way, we conduct extensive experiments on CIFAR-10, CIFAR-100, and ImageNette to demonstrate that our method achieves optimal robustness and exhibits generalization ability against unseen attacks.