Towards Adversarial Robustness via Transductive Learning
This work addresses adversarial robustness for machine learning models, but it is incremental as it builds on existing transductive learning approaches.
The paper tackles the problem of adversarial robustness by analyzing transductive learning defenses, proposing a new attack method that breaks previous defenses, and providing evidence for the utility of transductive learning in this context.
There has been emerging interest to use transductive learning for adversarial robustness (Goldwasser et al., NeurIPS 2020; Wu et al., ICML 2020). Compared to traditional "test-time" defenses, these defense mechanisms "dynamically retrain" the model based on test time input via transductive learning; and theoretically, attacking these defenses boils down to bilevel optimization, which seems to raise the difficulty for adaptive attacks. In this paper, we first formalize and analyze modeling aspects of transductive robustness. Then, we propose the principle of attacking model space for solving bilevel attack objectives, and present an instantiation of the principle which breaks previous transductive defenses. These attacks thus point to significant difficulties in the use of transductive learning to improve adversarial robustness. To this end, we present new theoretical and empirical evidence in support of the utility of transductive learning.