Mixing Classifiers to Alleviate the Accuracy-Robustness Trade-Off
This addresses the need for both high performance and rigorous robustness in safety-critical control systems, offering an incremental improvement over existing methods.
The paper tackles the accuracy-robustness trade-off in deep neural classifiers by proposing a method that mixes outputs from pre-trained standard and robust models, achieving noticeable improvement without additional training and providing theoretical guarantees against attacks within a certifiable radius.
Deep neural classifiers have recently found tremendous success in data-driven control systems. However, existing models suffer from a trade-off between accuracy and adversarial robustness. This limitation must be overcome in the control of safety-critical systems that require both high performance and rigorous robustness guarantees. In this work, we develop classifiers that simultaneously inherit high robustness from robust models and high accuracy from standard models. Specifically, we propose a theoretically motivated formulation that mixes the output probabilities of a standard neural network and a robust neural network. Both base classifiers are pre-trained, and thus our method does not require additional training. Our numerical experiments verify that the mixed classifier noticeably improves the accuracy-robustness trade-off and identify the confidence property of the robust base classifier as the key leverage of this more benign trade-off. Our theoretical results prove that under mild assumptions, when the robustness of the robust base model is certifiable, no alteration or attack within a closed-form $\ell_p$ radius on an input can result in the misclassification of the mixed classifier.