CARBEN: Composite Adversarial Robustness Benchmark
This work addresses the need for better robustness evaluation in machine learning, particularly for security-critical applications, but is incremental as it builds on existing adversarial attack methods.
The paper tackles the problem of evaluating adversarial robustness against composite adversarial attacks (CAA), which combine multiple threat models, and demonstrates how attack order affects results while providing real-time inferences and a leaderboard for benchmarking.
Prior literature on adversarial attack methods has mainly focused on attacking with and defending against a single threat model, e.g., perturbations bounded in Lp ball. However, multiple threat models can be combined into composite perturbations. One such approach, composite adversarial attack (CAA), not only expands the perturbable space of the image, but also may be overlooked by current modes of robustness evaluation. This paper demonstrates how CAA's attack order affects the resulting image, and provides real-time inferences of different models, which will facilitate users' configuration of the parameters of the attack level and their rapid evaluation of model prediction. A leaderboard to benchmark adversarial robustness against CAA is also introduced.