Certified Robustness to Adversarial Word Substitutions
This addresses the vulnerability of NLP models to adversarial attacks, providing certified robustness for tasks like sentiment analysis and natural language inference, though it is incremental in applying existing techniques to a specific domain.
The paper tackles the problem of adversarial word substitutions in NLP models by training the first models provably robust to all such substitutions, achieving 75% adversarial accuracy on sentiment analysis and natural language inference tasks, compared to 8% and 35% for baseline methods.
State-of-the-art NLP models can often be fooled by adversaries that apply seemingly innocuous label-preserving transformations (e.g., paraphrasing) to input text. The number of possible transformations scales exponentially with text length, so data augmentation cannot cover all transformations of an input. This paper considers one exponentially large family of label-preserving transformations, in which every word in the input can be replaced with a similar word. We train the first models that are provably robust to all word substitutions in this family. Our training procedure uses Interval Bound Propagation (IBP) to minimize an upper bound on the worst-case loss that any combination of word substitutions can induce. To evaluate models' robustness to these transformations, we measure accuracy on adversarially chosen word substitutions applied to test examples. Our IBP-trained models attain $75\%$ adversarial accuracy on both sentiment analysis on IMDB and natural language inference on SNLI. In comparison, on IMDB, models trained normally and ones trained with data augmentation achieve adversarial accuracy of only $8\%$ and $35\%$, respectively.