Perturbations in the Wild: Leveraging Human-Written Text Perturbations for Realistic Adversarial Attack and Defense
This work addresses the challenge of creating stealthy and effective adversarial attacks for text classification, which is important for improving model robustness in security and content moderation applications, though it is incremental as it builds on existing adversarial attack methods.
The authors tackled the problem of generating realistic adversarial text attacks by extracting over 600K human-written perturbations from real-world data, resulting in attacks that achieved 83% and 91% success rates on BERT and RoBERTa while improving semantic preservation and stealthiness by 50% and 40% over baselines.
We proposes a novel algorithm, ANTHRO, that inductively extracts over 600K human-written text perturbations in the wild and leverages them for realistic adversarial attack. Unlike existing character-based attacks which often deductively hypothesize a set of manipulation strategies, our work is grounded on actual observations from real-world texts. We find that adversarial texts generated by ANTHRO achieve the best trade-off between (1) attack success rate, (2) semantic preservation of the original text, and (3) stealthiness--i.e. indistinguishable from human writings hence harder to be flagged as suspicious. Specifically, our attacks accomplished around 83% and 91% attack success rates on BERT and RoBERTa, respectively. Moreover, it outperformed the TextBugger baseline with an increase of 50% and 40% in terms of semantic preservation and stealthiness when evaluated by both layperson and professional human workers. ANTHRO can further enhance a BERT classifier's performance in understanding different variations of human-written toxic texts via adversarial training when compared to the Perspective API.