Statistically Robust Neural Network Classification
This work addresses robustness assessment for neural networks in scenarios with probabilistic corruptions, providing a complementary measure to adversarial metrics, but it is incremental as it builds on existing risk frameworks.
The paper tackles the problem of measuring robustness in neural networks when test-time perturbations are probabilistic rather than adversarial, by developing a statistically robust risk (SRR) framework. The result shows that SRR offers simpler estimation and training than adversarial risks, with theoretical and experimental evidence of superior generalization on high-dimensional datasets.
Despite their numerous successes, there are many scenarios where adversarial risk metrics do not provide an appropriate measure of robustness. For example, test-time perturbations may occur in a probabilistic manner rather than being generated by an explicit adversary, while the poor train--test generalization of adversarial metrics can limit their usage to simple problems. Motivated by this, we develop a probabilistic robust risk framework, the statistically robust risk (SRR), which considers pointwise corruption distributions, as opposed to worst-case adversaries. The SRR provides a distinct and complementary measure of robust performance, compared to natural and adversarial risk. We show that the SRR admits estimation and training schemes which are as simple and efficient as for the natural risk: these simply require noising the inputs, but with a principled derivation for exactly how and why this should be done. Furthermore, we demonstrate both theoretically and experimentally that it can provide superior generalization performance compared with adversarial risks, enabling application to high-dimensional datasets.