A Novel Plug-and-Play Approach for Adversarially Robust Generalization
This work addresses the challenge of adversarial robustness for ML practitioners, offering a versatile framework applicable to multiple supervised and unsupervised problems, though it appears incremental as it builds on existing robust training methods.
The authors tackled the problem of making machine learning models robust to adversarial perturbations by proposing a plug-and-play framework that provides exact solutions for various loss functions and norm constraints, and derived new generalization bounds based on adversarial Rademacher complexity. They demonstrated the approach with sanity-check experiments on real-world datasets, reporting minimal computational overhead.
In this work, we propose a robust framework that employs adversarially robust training to safeguard the ML models against perturbed testing data. Our contributions can be seen from both computational and statistical perspectives. Firstly, from a computational/optimization point of view, we derive the ready-to-use exact solution for several widely used loss functions with a variety of norm constraints on adversarial perturbation for various supervised and unsupervised ML problems, including regression, classification, two-layer neural networks, graphical models, and matrix completion. The solutions are either in closed-form, or an easily tractable optimization problem such as 1-D convex optimization, semidefinite programming, difference of convex programming or a sorting-based algorithm. Secondly, from statistical/generalization viewpoint, using some of these results, we derive novel bounds of the adversarial Rademacher complexity for various problems, which entails new generalization bounds. Thirdly, we perform some sanity-check experiments on real-world datasets for supervised problems such as regression and classification, as well as for unsupervised problems such as matrix completion and learning graphical models, with very little computational overhead.