Randomness in ML Defenses Helps Persistent Attackers and Hinders Evaluators
This work addresses the challenge of evaluating and securing ML systems against adaptive adversaries, offering a practical framework for defense design.
The paper tackles the problem of designing robust ML defenses by arguing that white-box defenses should avoid randomness, showing that deterministic defenses simplify robustness evaluation without reducing effectiveness.
It is becoming increasingly imperative to design robust ML defenses. However, recent work has found that many defenses that initially resist state-of-the-art attacks can be broken by an adaptive adversary. In this work we take steps to simplify the design of defenses and argue that white-box defenses should eschew randomness when possible. We begin by illustrating a new issue with the deployment of randomized defenses that reduces their security compared to their deterministic counterparts. We then provide evidence that making defenses deterministic simplifies robustness evaluation, without reducing the effectiveness of a truly robust defense. Finally, we introduce a new defense evaluation framework that leverages a defense's deterministic nature to better evaluate its adversarial robustness.