Elephant in the Room: An Evaluation Framework for Assessing Adversarial Examples in NLP
This work addresses the challenge of rigorous evaluation for adversarial examples in NLP, which is crucial for researchers and practitioners in security and robustness, though it is incremental as it builds on existing methods.
The paper tackles the problem of evaluating adversarial examples in NLP by proposing a framework with automatic metrics and human guidelines to assess quality based on meaning, readability, and classification impact, finding that some methods produce poor-quality examples and that factors like text length and classifier architecture influence performance.
An adversarial example is an input transformed by small perturbations that machine learning models consistently misclassify. While there are a number of methods proposed to generate adversarial examples for text data, it is not trivial to assess the quality of these adversarial examples, as minor perturbations (such as changing a word in a sentence) can lead to a significant shift in their meaning, readability and classification label. In this paper, we propose an evaluation framework consisting of a set of automatic evaluation metrics and human evaluation guidelines, to rigorously assess the quality of adversarial examples based on the aforementioned properties. We experiment with six benchmark attacking methods and found that some methods generate adversarial examples with poor readability and content preservation. We also learned that multiple factors could influence the attacking performance, such as the length of the text inputs and architecture of the classifiers.