Model-Agnostic Meta-Attack: Towards Reliable Evaluation of Adversarial Robustness
This work addresses the need for more reliable evaluation methods in adversarial machine learning, particularly for researchers and practitioners developing defenses, though it is incremental as it builds on existing attack frameworks.
The paper tackles the problem of insufficient robustness evaluations for deep neural networks against adversarial examples by proposing a Model-Agnostic Meta-Attack (MAMA) approach that automatically discovers stronger attack algorithms, leading to more reliable adversarial robustness assessments with consistent performance improvements across various defenses.
The vulnerability of deep neural networks to adversarial examples has motivated an increasing number of defense strategies for promoting model robustness. However, the progress is usually hampered by insufficient robustness evaluations. As the de facto standard to evaluate adversarial robustness, adversarial attacks typically solve an optimization problem of crafting adversarial examples with an iterative process. In this work, we propose a Model-Agnostic Meta-Attack (MAMA) approach to discover stronger attack algorithms automatically. Our method learns the optimizer in adversarial attacks parameterized by a recurrent neural network, which is trained over a class of data samples and defenses to produce effective update directions during adversarial example generation. Furthermore, we develop a model-agnostic training algorithm to improve the generalization ability of the learned optimizer when attacking unseen defenses. Our approach can be flexibly incorporated with various attacks and consistently improves the performance with little extra computational cost. Extensive experiments demonstrate the effectiveness of the learned attacks by MAMA compared to the state-of-the-art attacks on different defenses, leading to a more reliable evaluation of adversarial robustness.