The Consistency of Adversarial Training for Binary Classification
This addresses the need for theoretical guarantees in adversarial robustness, but it is incremental as it builds on existing consistency frameworks.
The paper tackles the problem of ensuring statistical consistency for adversarial training in binary classification, showing which supremum-based surrogates are consistent for certain distributions and providing quantitative bounds on adversarial risks.
Robustness to adversarial perturbations is of paramount concern in modern machine learning. One of the state-of-the-art methods for training robust classifiers is adversarial training, which involves minimizing a supremum-based surrogate risk. The statistical consistency of surrogate risks is well understood in the context of standard machine learning, but not in the adversarial setting. In this paper, we characterize which supremum-based surrogates are consistent for distributions absolutely continuous with respect to Lebesgue measure in binary classification. Furthermore, we obtain quantitative bounds relating adversarial surrogate risks to the adversarial classification risk. Lastly, we discuss implications for the $\cH$-consistency of adversarial training.