Leveraging Extracted Model Adversaries for Improved Black Box Attacks
This addresses security vulnerabilities in black box AI systems, but it is incremental as it builds on existing model extraction and attack methods.
The paper tackles adversarial attacks on black box question answering models by first extracting an approximate model and then generating perturbations to cause failures, resulting in a 25% F1 improvement over a white box attack and 11% F1 improvement over a black box attack.
We present a method for adversarial input generation against black box models for reading comprehension based question answering. Our approach is composed of two steps. First, we approximate a victim black box model via model extraction (Krishna et al., 2020). Second, we use our own white box method to generate input perturbations that cause the approximate model to fail. These perturbed inputs are used against the victim. In experiments we find that our method improves on the efficacy of the AddAny---a white box attack---performed on the approximate model by 25% F1, and the AddSent attack---a black box attack---by 11% F1 (Jia and Liang, 2017).