Distributional Adversarial Loss
This work addresses adversarial robustness in machine learning by proposing a novel theoretical framework that bridges and extends existing approaches, offering potential for improved defense mechanisms.
The paper introduces distributional adversarial loss, a new adversarial robustness notion where each example's perturbation set is a family of distributions, and provides PAC-learning sample complexity bounds for it, unifying randomized smoothing and robust learning with sample complexity results.
We initiate the study of a new notion of adversarial loss which we call distributional adversarial loss. In this notion, we assume for each original example, the allowed adversarial perturbation set is a family of distributions, and the adversarial loss over each example is the maximum loss over all the associated distributions. The goal is to minimize the overall adversarial loss. We show sample complexity bounds in the PAC-learning setting for our notion of adversarial loss. Our notion of adversarial loss contrasts the prior work on robust learning that considers a set of points, not distributions, as the perturbation set of each clean example. As an application of our approach, we show how to unify the two lines of work on randomized smoothing and robust learning in the PAC-learning setting and derive sample complexity bounds for randomized smoothing methods. Furthermore, we investigate the role of randomness in achieving robustness against adversarial attacks. We show a general derandomization technique that preserves the extent of a randomized classifier's robustness against adversarial attacks and show its effectiveness empirically.