What Learned Representations and Influence Functions Can Tell Us About Adversarial Examples
This work addresses the challenge of adversarial example detection in NLP, which is crucial for improving model robustness in applications like text classification and sentiment analysis, though it is incremental as it adapts existing methods from another domain.
The paper tackled the problem of detecting adversarial examples in NLP by adapting techniques from image processing, specifically using nearest neighbors with influence functions and Mahalanobis distances, resulting in a state-of-the-art detector that outperformed strong baselines.
Adversarial examples, deliberately crafted using small perturbations to fool deep neural networks, were first studied in image processing and more recently in NLP. While approaches to detecting adversarial examples in NLP have largely relied on search over input perturbations, image processing has seen a range of techniques that aim to characterise adversarial subspaces over the learned representations. In this paper, we adapt two such approaches to NLP, one based on nearest neighbors and influence functions and one on Mahalanobis distances. The former in particular produces a state-of-the-art detector when compared against several strong baselines; moreover, the novel use of influence functions provides insight into how the nature of adversarial example subspaces in NLP relate to those in image processing, and also how they differ depending on the kind of NLP task.