Certifying Some Distributional Robustness with Principled Adversarial Training
This provides a principled defense against adversarial attacks for neural network users, though it is incremental as it builds on existing robust optimization frameworks.
The paper tackles the vulnerability of neural networks to adversarial examples by using distributionally robust optimization to guarantee performance under input perturbations, achieving moderate robustness with low computational cost and matching or outperforming heuristic methods for imperceptible perturbations.
Neural networks are vulnerable to adversarial examples and researchers have proposed many heuristic attack and defense mechanisms. We address this problem through the principled lens of distributionally robust optimization, which guarantees performance under adversarial input perturbations. By considering a Lagrangian penalty formulation of perturbing the underlying data distribution in a Wasserstein ball, we provide a training procedure that augments model parameter updates with worst-case perturbations of training data. For smooth losses, our procedure provably achieves moderate levels of robustness with little computational or statistical cost relative to empirical risk minimization. Furthermore, our statistical guarantees allow us to efficiently certify robustness for the population loss. For imperceptible perturbations, our method matches or outperforms heuristic approaches.