Globally-Robust Neural Networks
This addresses the threat of adversarial attacks for machine learning practitioners by providing a more efficient and certifiably robust method, though it builds incrementally on existing robustness techniques.
The paper tackles the problem of adversarial examples by formalizing global robustness for neural networks, which enables efficient on-line certification and robust training, achieving state-of-the-art verifiable accuracy with significantly reduced time and memory costs, such as training a large robust Tiny-Imagenet model in hours.
The threat of adversarial examples has motivated work on training certifiably robust neural networks to facilitate efficient verification of local robustness at inference time. We formalize a notion of global robustness, which captures the operational properties of on-line local robustness certification while yielding a natural learning objective for robust training. We show that widely-used architectures can be easily adapted to this objective by incorporating efficient global Lipschitz bounds into the network, yielding certifiably-robust models by construction that achieve state-of-the-art verifiable accuracy. Notably, this approach requires significantly less time and memory than recent certifiable training methods, and leads to negligible costs when certifying points on-line; for example, our evaluation shows that it is possible to train a large robust Tiny-Imagenet model in a matter of hours. Our models effectively leverage inexpensive global Lipschitz bounds for real-time certification, despite prior suggestions that tighter local bounds are needed for good performance; we posit this is possible because our models are specifically trained to achieve tighter global bounds. Namely, we prove that the maximum achievable verifiable accuracy for a given dataset is not improved by using a local bound.