Architectural Adversarial Robustness: The Case for Deep Pursuit
This paper addresses the problem of adversarial robustness in deep neural networks, which is a significant concern for the reliability and security of AI systems.
Deep neural networks are vulnerable to adversarial noise. This paper proposes deep pursuit, a method that reframes network activations as a single global optimization problem to improve robustness.
Despite their unmatched performance, deep neural networks remain susceptible to targeted attacks by nearly imperceptible levels of adversarial noise. While the underlying cause of this sensitivity is not well understood, theoretical analyses can be simplified by reframing each layer of a feed-forward network as an approximate solution to a sparse coding problem. Iterative solutions using basis pursuit are theoretically more stable and have improved adversarial robustness. However, cascading layer-wise pursuit implementations suffer from error accumulation in deeper networks. In contrast, our new method of deep pursuit approximates the activations of all layers as a single global optimization problem, allowing us to consider deeper, real-world architectures with skip connections such as residual networks. Experimentally, our approach demonstrates improved robustness to adversarial noise.