Towards Generalized Certified Robustness with Multi-Norm Training
This work addresses the problem of limited robustness in certified models by enabling multi-norm robustness, which is a step towards generalized certified robustness for safety-critical applications.
Existing certified training methods are limited to single perturbation types (e.g., l∞ or l2), leading to poor robustness against other perturbations. The authors propose CURE, the first multi-norm certified training framework, which improves union robustness across multiple perturbation types, achieving 32.0% on MNIST, 25.8% on CIFAR-10, and 10.6% on TinyImagenet, and also generalizes to unseen geometric and patch perturbations.
Existing certified training methods can only train models to be robust against a certain perturbation type (e.g. $l_\infty$ or $l_2$). However, an $l_\infty$ certifiably robust model may not be certifiably robust against $l_2$ perturbation (and vice versa) and also has low robustness against other perturbations (e.g. geometric and patch transformation). By constructing a theoretical framework to analyze and mitigate the tradeoff, we propose the first multi-norm certified training framework \textbf{CURE}, consisting of several multi-norm certified training methods, to attain better \emph{union robustness} when training from scratch or fine-tuning a pre-trained certified model. Inspired by our theoretical findings, we devise bound alignment and connect natural training with certified training for better union robustness. Compared with SOTA-certified training, \textbf{CURE} improves union robustness to $32.0\%$ on MNIST, $25.8\%$ on CIFAR-10, and $10.6\%$ on TinyImagenet across different epsilon values. It leads to better generalization on a diverse set of challenging unseen geometric and patch perturbations to $6.8\%$ and $16.0\%$ on CIFAR-10. Overall, our contributions pave a path towards \textit{generalized certified robustness}.