Adversarial Examples Are Not Bugs, They Are Superposition
This addresses a core, unresolved issue in deep learning for researchers and practitioners, offering a novel explanation that could shift understanding and defense strategies.
The paper tackles the problem of adversarial examples in deep learning by proposing superposition as a fundamental mechanism, providing theoretical and experimental evidence across toy models and ResNet18 to support this hypothesis.
Adversarial examples -- inputs with imperceptible perturbations that fool neural networks -- remain one of deep learning's most perplexing phenomena despite nearly a decade of research. While numerous defenses and explanations have been proposed, there is no consensus on the fundamental mechanism. One underexplored hypothesis is that superposition, a concept from mechanistic interpretability, may be a major contributing factor, or even the primary cause. We present four lines of evidence in support of this hypothesis, greatly extending prior arguments by Elhage et al. (2022): (1) superposition can theoretically explain a range of adversarial phenomena, (2) in toy models, intervening on superposition controls robustness, (3) in toy models, intervening on robustness (via adversarial training) controls superposition, and (4) in ResNet18, intervening on robustness (via adversarial training) controls superposition.