The best defense is a good offense: Countering black box attacks by predicting slightly wrong labels
This addresses security vulnerabilities in machine learning models for applications requiring robust defense against adversarial attacks, though it is incremental as it builds on existing counter-attack strategies.
The paper tackles the problem of black-box attacks via model theft by introducing a defense method that perturbs output label distributions to disrupt substitute model training, demonstrating that this approach effectively prevents attacks without compromising normal model performance.
Black-Box attacks on machine learning models occur when an attacker, despite having no access to the inner workings of a model, can successfully craft an attack by means of model theft. The attacker will train an own substitute model that mimics the model to be attacked. The substitute can then be used to design attacks against the original model, for example by means of adversarial samples. We put ourselves in the shoes of the defender and present a method that can successfully avoid model theft by mounting a counter-attack. Specifically, to any incoming query, we slightly perturb our output label distribution in a way that makes substitute training infeasible. We demonstrate that the perturbation does not affect the ordinary use of our model, but results in an effective defense against attacks based on model theft.