Average Margin Regularization for Classifiers
This addresses the problem of adversarial vulnerability in machine learning models, offering a method to enhance robustness without sacrificing accuracy, though it appears incremental relative to existing regularization techniques.
The paper tackles the tradeoff between classification accuracy and adversarial robustness by proposing average margin regularization, which theoretically and empirically improves both accuracy and robustness for classifiers like SVMs, with empirical demonstrations on synthetic and real data.
Adversarial robustness has become an important research topic given empirical demonstrations on the lack of robustness of deep neural networks. Unfortunately, recent theoretical results suggest that adversarial training induces a strict tradeoff between classification accuracy and adversarial robustness. In this paper, we propose and then study a new regularization for any margin classifier or deep neural network. We motivate this regularization by a novel generalization bound that shows a tradeoff in classifier accuracy between maximizing its margin and average margin. We thus call our approach an average margin (AM) regularization, and it consists of a linear term added to the objective. We theoretically show that for certain distributions AM regularization can both improve classifier accuracy and robustness to adversarial attacks. We conclude by using both synthetic and real data to empirically show that AM regularization can strictly improve both accuracy and robustness for support vector machine's (SVM's), relative to unregularized classifiers and adversarially trained classifiers.