Classifier-independent Lower-Bounds for Adversarial Robustness
This work addresses the fundamental limits of adversarial robustness in machine learning, offering theoretical insights applicable across all classifiers, which is foundational rather than incremental.
The paper tackles the problem of deriving universal lower bounds on adversarial robustness for any classifier, using optimal transport theory to compute the Bayes-optimal error under attacks and providing explicit bounds for distance-based attacks based on data geometry.
We theoretically analyse the limits of robustness to test-time adversarial and noisy examples in classification. Our work focuses on deriving bounds which uniformly apply to all classifiers (i.e all measurable functions from features to labels) for a given problem. Our contributions are two-fold. (1) We use optimal transport theory to derive variational formulae for the Bayes-optimal error a classifier can make on a given classification problem, subject to adversarial attacks. The optimal adversarial attack is then an optimal transport plan for a certain binary cost-function induced by the specific attack model, and can be computed via a simple algorithm based on maximal matching on bipartite graphs. (2) We derive explicit lower-bounds on the Bayes-optimal error in the case of the popular distance-based attacks. These bounds are universal in the sense that they depend on the geometry of the class-conditional distributions of the data, but not on a particular classifier. Our results are in sharp contrast with the existing literature, wherein adversarial vulnerability of classifiers is derived as a consequence of nonzero ordinary test error.