Understanding the Error in Evaluating Adversarial Robustness
This work addresses the critical problem of accurately evaluating adversarial robustness for the realistic deployment of deep models, which is important for researchers and practitioners in AI security. It is an incremental step in understanding existing issues.
The paper investigates the discrepancy between true and evaluated adversarial robustness in deep neural networks, identifying 'gradient traps' as a key phenomenon leading to incompetent adversaries. It decomposes this evaluation error into three components, each stemming from a specific compromise, and offers evaluation suggestions. Experiments show that this error exists empirically and that current adversarial defenses remain vulnerable.
Deep neural networks are easily misled by adversarial examples. Although lots of defense methods are proposed, many of them are demonstrated to lose effectiveness when against properly performed adaptive attacks. How to evaluate the adversarial robustness effectively is important for the realistic deployment of deep models, but yet still unclear. To provide a reasonable solution, one of the primary things is to understand the error (or gap) between the true adversarial robustness and the evaluated one, what is it and why it exists. Several works are done in this paper to make it clear. Firstly, we introduce an interesting phenomenon named gradient traps, which lead to incompetent adversaries and are demonstrated to be a manifestation of evaluation error. Then, we analyze the error and identify that there are three components. Each of them is caused by a specific compromise. Moreover, based on the above analysis, we present our evaluation suggestions. Experiments on adversarial training and its variations indicate that: (1) the error does exist empirically, and (2) these defenses are still vulnerable. We hope these analyses and results will help the community to develop more powerful defenses.