Solving adversarial examples requires solving exponential misalignment
This work addresses the persistent issue of adversarial attacks in machine learning by linking it to fundamental misalignment between machine and human perception, suggesting a major impediment to robustness.
The paper tackles the problem of adversarial examples by analyzing the perceptual manifolds (PMs) of neural networks, finding that their dimensionalities are orders of magnitude higher than human concepts, leading to exponential misalignment. This geometric hypothesis predicts that adversarial robustness requires dimensional alignment, and experiments across 18 networks confirm negative correlations between robust accuracy and PM dimension, with even robust networks showing exponential misalignment.
Adversarial attacks - input perturbations imperceptible to humans that fool neural networks - remain both a persistent failure mode in machine learning, and a phenomenon with mysterious origins. To shed light, we define and analyze a network's perceptual manifold (PM) for a class concept as the space of all inputs confidently assigned to that class by the network. We find, strikingly, that the dimensionalities of neural network PMs are orders of magnitude higher than those of natural human concepts. Since volume typically grows exponentially with dimension, this suggests exponential misalignment between machines and humans, with exponentially many inputs confidently assigned to concepts by machines but not humans. Furthermore, this provides a natural geometric hypothesis for the origin of adversarial examples: because a network's PM fills such a large region of input space, any input will be very close to any class concept's PM. Our hypothesis thus suggests that adversarial robustness cannot be attained without dimensional alignment of machine and human PMs, and therefore makes strong predictions: both robust accuracy and distance to any PM should be negatively correlated with the PM dimension. We confirmed these predictions across 18 different networks of varying robust accuracy. Crucially, we find even the most robust networks are still exponentially misaligned, and only the few PMs whose dimensionality approaches that of human concepts exhibit alignment to human perception. Our results connect the fields of alignment and adversarial examples, and suggest the curse of high dimensionality of machine PMs is a major impediment to adversarial robustness.