Searching for an Effective Defender: Benchmarking Defense against Adversarial Word Substitution
This work addresses the problem of evaluating and improving robustness in neural text classifiers for researchers and practitioners, though it is incremental as it builds on existing defense methods.
The paper tackled the lack of systematic comparison of defense methods against adversarial word-substitution attacks in NLP by conducting comprehensive research and proposing a new method that achieved the highest accuracy on clean and adversarial examples on AGNEWS and IMDB datasets by a significant margin.
Recent studies have shown that deep neural networks are vulnerable to intentionally crafted adversarial examples, and various methods have been proposed to defend against adversarial word-substitution attacks for neural NLP models. However, there is a lack of systematic study on comparing different defense approaches under the same attacking setting. In this paper, we seek to fill the gap of systematic studies through comprehensive researches on understanding the behavior of neural text classifiers trained by various defense methods under representative adversarial attacks. In addition, we propose an effective method to further improve the robustness of neural text classifiers against such attacks and achieved the highest accuracy on both clean and adversarial examples on AGNEWS and IMDB datasets by a significant margin.