Understanding Intrinsic Robustness Using Label Uncertainty
This work addresses the fundamental issue of whether robust classifiers exist for specific tasks, offering a more precise characterization that could impact adversarial robustness research, though it appears incremental by building on existing concentration methods.
The paper tackles the problem of measuring intrinsic robustness in adversarial machine learning by introducing label uncertainty, showing that error regions of state-of-the-art models have higher label uncertainty than random subsets, and adapting a concentration estimation algorithm to provide more accurate robustness measures for image classification benchmarks.
A fundamental question in adversarial machine learning is whether a robust classifier exists for a given task. A line of research has made some progress towards this goal by studying the concentration of measure, but we argue standard concentration fails to fully characterize the intrinsic robustness of a classification problem since it ignores data labels which are essential to any classification task. Building on a novel definition of label uncertainty, we empirically demonstrate that error regions induced by state-of-the-art models tend to have much higher label uncertainty than randomly-selected subsets. This observation motivates us to adapt a concentration estimation algorithm to account for label uncertainty, resulting in more accurate intrinsic robustness measures for benchmark image classification problems.