Data Poisoning against Differentially-Private Learners: Attacks and Defenses
This addresses security vulnerabilities in privacy-preserving ML for practitioners, but it is incremental as it builds on known attack and defense methods.
The paper tackles data poisoning attacks on differentially-private machine learning models, showing that while such learners resist attacks with few poisoned items, protection degrades as more data is poisoned, with experiments demonstrating effective attacks when sufficient items are compromised.
Data poisoning attacks aim to manipulate the model produced by a learning algorithm by adversarially modifying the training set. We consider differential privacy as a defensive measure against this type of attack. We show that such learners are resistant to data poisoning attacks when the adversary is only able to poison a small number of items. However, this protection degrades as the adversary poisons more data. To illustrate, we design attack algorithms targeting objective and output perturbation learners, two standard approaches to differentially-private machine learning. Experiments show that our methods are effective when the attacker is allowed to poison sufficiently many training items.