RoBIC: A benchmark suite for assessing classifiers robustness
This provides a standardized tool for researchers and practitioners to evaluate image classifier robustness, though it is incremental as it builds on existing benchmarking efforts.
The paper tackles the need for objective evaluation of classifier robustness by proposing RoBIC, a parameter-free benchmark that uses a half-distortion measure to assess robustness against attacks, showing significant differences in 16 models.
Many defenses have emerged with the development of adversarial attacks. Models must be objectively evaluated accordingly. This paper systematically tackles this concern by proposing a new parameter-free benchmark we coin RoBIC. RoBIC fairly evaluates the robustness of image classifiers using a new half-distortion measure. It gauges the robustness of the network against white and black box attacks, independently of its accuracy. RoBIC is faster than the other available benchmarks. We present the significant differences in the robustness of 16 recent models as assessed by RoBIC.