Reevaluating Adversarial Examples in Natural Language
This work addresses the need for more rigorous evaluation of adversarial examples in natural language processing, though it is incremental in refining existing attack frameworks.
The paper tackled the problem of inconsistent definitions for successful adversarial attacks in NLP, finding that state-of-the-art synonym substitution attacks often fail to preserve semantics and grammar, with attack success rates dropping over 70% when constraints are adjusted.
State-of-the-art attacks on NLP models lack a shared definition of a what constitutes a successful attack. We distill ideas from past work into a unified framework: a successful natural language adversarial example is a perturbation that fools the model and follows some linguistic constraints. We then analyze the outputs of two state-of-the-art synonym substitution attacks. We find that their perturbations often do not preserve semantics, and 38% introduce grammatical errors. Human surveys reveal that to successfully preserve semantics, we need to significantly increase the minimum cosine similarities between the embeddings of swapped words and between the sentence encodings of original and perturbed sentences.With constraints adjusted to better preserve semantics and grammaticality, the attack success rate drops by over 70 percentage points.