When adversarial examples are excusable
This work addresses the problem of making adversarial errors more understandable and less severe for neural network reliability, though it is incremental as it builds on existing adversarial example research.
The paper investigates adversarial errors in neural networks, finding that when adversarial examples are constrained to the data manifold or training includes Gaussian noise, adversarial errors are reduced by 90% and become similar to test errors near decision boundaries, rather than obvious mistakes.
Neural networks work remarkably well in practice and theoretically they can be universal approximators. However, they still make mistakes and a specific type of them called adversarial errors seem inexcusable to humans. In this work, we analyze both test errors and adversarial errors on a well controlled but highly non-linear visual classification problem. We find that, when approximating training on infinite data, test errors tend to be close to the ground truth decision boundary. Qualitatively speaking these are also more difficult for a human. By contrast, adversarial examples can be found almost everywhere and are often obvious mistakes. However, when we constrain adversarial examples to the manifold, we observe a 90\% reduction in adversarial errors. If we inflate the manifold by training with Gaussian noise we observe a similar effect. In both cases, the remaining adversarial errors tend to be close to the ground truth decision boundary. Qualitatively, the remaining adversarial errors are similar to test errors on difficult examples. They do not have the customary quality of being inexcusable mistakes.