Poisons that are learned faster are more effective
This work addresses the practical limitations of data privacy protection methods for machine learning practitioners, showing incremental insights into defense mechanisms.
The paper investigates the vulnerability of dataset poisoning attacks to early-stopping defenses, finding that poisons that achieve low training loss faster result in lower peak test accuracy and that a state-of-the-art poison is 7 times less effective when stopped early.
Imperceptible poisoning attacks on entire datasets have recently been touted as methods for protecting data privacy. However, among a number of defenses preventing the practical use of these techniques, early-stopping stands out as a simple, yet effective defense. To gauge poisons' vulnerability to early-stopping, we benchmark error-minimizing, error-maximizing, and synthetic poisons in terms of peak test accuracy over 100 epochs and make a number of surprising observations. First, we find that poisons that reach a low training loss faster have lower peak test accuracy. Second, we find that a current state-of-the-art error-maximizing poison is 7 times less effective when poison training is stopped at epoch 8. Third, we find that stronger, more transferable adversarial attacks do not make stronger poisons. We advocate for evaluating poisons in terms of peak test accuracy.