AdvRush: Searching for Adversarially Robust Neural Architectures
This addresses the vulnerability of deep neural networks to adversarial attacks, offering a novel approach beyond robust training methods, though it is incremental as it builds on existing neural architecture search techniques.
The paper tackles the problem of designing neural architectures with high intrinsic robustness against adversarial examples, proposing AdvRush, a neural architecture search algorithm that discovers architectures with smoother input loss landscapes, achieving 55.91% robust accuracy on CIFAR-10 under FGSM attack and 50.04% under AutoAttack after adversarial training.
Deep neural networks continue to awe the world with their remarkable performance. Their predictions, however, are prone to be corrupted by adversarial examples that are imperceptible to humans. Current efforts to improve the robustness of neural networks against adversarial examples are focused on developing robust training methods, which update the weights of a neural network in a more robust direction. In this work, we take a step beyond training of the weight parameters and consider the problem of designing an adversarially robust neural architecture with high intrinsic robustness. We propose AdvRush, a novel adversarial robustness-aware neural architecture search algorithm, based upon a finding that independent of the training method, the intrinsic robustness of a neural network can be represented with the smoothness of its input loss landscape. Through a regularizer that favors a candidate architecture with a smoother input loss landscape, AdvRush successfully discovers an adversarially robust neural architecture. Along with a comprehensive theoretical motivation for AdvRush, we conduct an extensive amount of experiments to demonstrate the efficacy of AdvRush on various benchmark datasets. Notably, on CIFAR-10, AdvRush achieves 55.91% robust accuracy under FGSM attack after standard training and 50.04% robust accuracy under AutoAttack after 7-step PGD adversarial training.