Phantom Transfer: Data Poisoning can Survive Data-Level Defences
For ML security researchers, this paper provides an existence proof that current data-level defenses are insufficient against advanced poisoning attacks, highlighting the need for white-box and post-training defenses.
Phantom Transfer is a data poisoning attack that survives 11 data-level defenses, including paraphrasing, by embedding poison that cannot be filtered out even if its placement is known. It demonstrates that maximum-affordance defenses can fail against sophisticated attacks.
We present a data poisoning attack -- Phantom Transfer -- with the property that, even if you know precisely how the poison was placed into an otherwise benign dataset, you cannot filter it out. We achieve this by modifying subliminal learning to work in real-world contexts and demonstrate that the attack works regardless of which model produced the data, which model is trained on the data or what the attack target is. Furthermore, the attack survives 11 tested data-level defences, including one where every sample is paraphrased by another model. We characterise when this attack works best and show that it can be used to plant password-triggered behaviours into models while still beating defences. In short, we provide an existence proof that maximum-affordance defences can fail to stop sophisticated data poisoning attacks. We suggest that future defences should be supplemented with white-box methods and post-training model audits.