Targeted Adversarial Training for Natural Language Understanding
This work addresses adversarial robustness in natural language understanding, which is crucial for deploying reliable AI systems in real-world applications, and it is incremental as it builds upon standard adversarial training methods.
The paper tackles the problem of improving adversarial training for natural language understanding by introducing a Targeted Adversarial Training (TAT) algorithm that prioritizes training on model errors, resulting in significant accuracy gains on GLUE and new state-of-the-art zero-shot results on XNLI.
We present a simple yet effective Targeted Adversarial Training (TAT) algorithm to improve adversarial training for natural language understanding. The key idea is to introspect current mistakes and prioritize adversarial training steps to where the model errs the most. Experiments show that TAT can significantly improve accuracy over standard adversarial training on GLUE and attain new state-of-the-art zero-shot results on XNLI. Our code will be released at: https://github.com/namisan/mt-dnn.