Step by Step Loss Goes Very Far: Multi-Step Quantization for Adversarial Text Attacks
This addresses the problem of adversarial robustness for NLP practitioners, but it appears incremental as it builds on existing gradient-based attack methods.
The paper tackles the problem of generating adversarial examples against transformer-based language models by proposing a gradient-based attack that searches in a continuous token probability space, using a multi-step quantization method to bridge the gap between continuous and discrete representations. The result is a method that significantly outperforms other approaches on various NLP tasks, though no concrete numbers are provided in the abstract.
We propose a novel gradient-based attack against transformer-based language models that searches for an adversarial example in a continuous space of token probabilities. Our algorithm mitigates the gap between adversarial loss for continuous and discrete text representations by performing multi-step quantization in a quantization-compensation loop. Experiments show that our method significantly outperforms other approaches on various natural language processing (NLP) tasks.