RayS: A Ray Searching Method for Hard-label Adversarial Attack
This work addresses the challenge of evaluating and attacking deep neural networks under practical hard-label settings, potentially exposing falsely robust models in security-critical applications.
The paper tackles the problem of hard-label adversarial attacks, where only the prediction label is accessible, by introducing RayS, a method that reformulates the attack as a discrete search problem, eliminating the need for gradient estimation and reducing query complexity. The result is a significant improvement in attack success rate and efficiency, with the method also revealing that some claimed robust models are vulnerable, showing up to a 28% drop in robust accuracy compared to standard attacks.
Deep neural networks are vulnerable to adversarial attacks. Among different attack settings, the most challenging yet the most practical one is the hard-label setting where the attacker only has access to the hard-label output (prediction label) of the target model. Previous attempts are neither effective enough in terms of attack success rate nor efficient enough in terms of query complexity under the widely used $L_\infty$ norm threat model. In this paper, we present the Ray Searching attack (RayS), which greatly improves the hard-label attack effectiveness as well as efficiency. Unlike previous works, we reformulate the continuous problem of finding the closest decision boundary into a discrete problem that does not require any zeroth-order gradient estimation. In the meantime, all unnecessary searches are eliminated via a fast check step. This significantly reduces the number of queries needed for our hard-label attack. Moreover, interestingly, we found that the proposed RayS attack can also be used as a sanity check for possible "falsely robust" models. On several recently proposed defenses that claim to achieve the state-of-the-art robust accuracy, our attack method demonstrates that the current white-box/black-box attacks could still give a false sense of security and the robust accuracy drop between the most popular PGD attack and RayS attack could be as large as $28\%$. We believe that our proposed RayS attack could help identify falsely robust models that beat most white-box/black-box attacks.