Robust Principles: Architectural Design Principles for Adversarially Robust CNNs
This work addresses the challenge of designing adversarially robust CNNs for machine learning practitioners, though it is incremental as it synthesizes and validates existing ideas rather than introducing a new paradigm.
The research tackled the problem of unifying diverging opinions on how architectural components affect adversarial robustness in CNNs, resulting in a suite of three design principles that consistently improved AutoAttack accuracy by 1-3 percentage points on CIFAR-10/CIFAR-100 and 4-9 percentage points on ImageNet.
Our research aims to unify existing works' diverging opinions on how architectural components affect the adversarial robustness of CNNs. To accomplish our goal, we synthesize a suite of three generalizable robust architectural design principles: (a) optimal range for depth and width configurations, (b) preferring convolutional over patchify stem stage, and (c) robust residual block design through adopting squeeze and excitation blocks and non-parametric smooth activation functions. Through extensive experiments across a wide spectrum of dataset scales, adversarial training methods, model parameters, and network design spaces, our principles consistently and markedly improve AutoAttack accuracy: 1-3 percentage points (pp) on CIFAR-10 and CIFAR-100, and 4-9 pp on ImageNet. The code is publicly available at https://github.com/poloclub/robust-principles.