What Doesn't Kill You Makes You Robust(er): How to Adversarially Train against Data Poisoning
This addresses the security challenge of data poisoning for machine learning systems, offering a robust defense that overcomes flaws in existing methods.
The paper tackles the problem of defending against data poisoning attacks by extending adversarial training to create and inject poisons during training, showing that this method withstands adaptive attacks, generalizes to diverse threat models, and incurs a better performance trade-off than previous defenses like DP-SGD or evasion adversarial training.
Data poisoning is a threat model in which a malicious actor tampers with training data to manipulate outcomes at inference time. A variety of defenses against this threat model have been proposed, but each suffers from at least one of the following flaws: they are easily overcome by adaptive attacks, they severely reduce testing performance, or they cannot generalize to diverse data poisoning threat models. Adversarial training, and its variants, are currently considered the only empirically strong defense against (inference-time) adversarial attacks. In this work, we extend the adversarial training framework to defend against (training-time) data poisoning, including targeted and backdoor attacks. Our method desensitizes networks to the effects of such attacks by creating poisons during training and injecting them into training batches. We show that this defense withstands adaptive attacks, generalizes to diverse threat models, and incurs a better performance trade-off than previous defenses such as DP-SGD or (evasion) adversarial training.