RAMP: Boosting Adversarial Robustness Against Multiple $l_p$ Perturbations for Universal Robustness
This work addresses the need for models robust against diverse real-world adversarial attacks, which is crucial for practical AI security, though it is incremental as it builds on existing adversarial training methods.
The paper tackles the problem of low multiple-norm robustness in adversarial training by proposing RAMP, a framework that improves union accuracy against multiple l_p perturbations, achieving up to 53.3% on CIFAR-10 and 29.1% on ImageNet in robust fine-tuning, and 44.6% union accuracy with 81.2% clean accuracy on CIFAR-10 when training from scratch.
Most existing works focus on improving robustness against adversarial attacks bounded by a single $l_p$ norm using adversarial training (AT). However, these AT models' multiple-norm robustness (union accuracy) is still low, which is crucial since in the real-world an adversary is not necessarily bounded by a single norm. The tradeoffs among robustness against multiple $l_p$ perturbations and accuracy/robustness make obtaining good union and clean accuracy challenging. We design a logit pairing loss to improve the union accuracy by analyzing the tradeoffs from the lens of distribution shifts. We connect natural training (NT) with AT via gradient projection, to incorporate useful information from NT into AT, where we empirically and theoretically show it moderates the accuracy/robustness tradeoff. We propose a novel training framework \textbf{RAMP}, to boost the robustness against multiple $l_p$ perturbations. \textbf{RAMP} can be easily adapted for robust fine-tuning and full AT. For robust fine-tuning, \textbf{RAMP} obtains a union accuracy up to $53.3\%$ on CIFAR-10, and $29.1\%$ on ImageNet. For training from scratch, \textbf{RAMP} achieves a union accuracy of $44.6\%$ and good clean accuracy of $81.2\%$ on ResNet-18 against AutoAttack on CIFAR-10. Beyond multi-norm robustness \textbf{RAMP}-trained models achieve superior \textit{universal robustness}, effectively generalizing against a range of unseen adversaries and natural corruptions.