Not All Samples Are Equal: Quantifying Instance-level Difficulty in Targeted Data Poisoning
This work addresses a specific threat model in machine learning security by providing insights for practitioners to assess vulnerability, though it is incremental in nature.
The paper tackles the problem of understanding why certain test samples are more vulnerable to targeted data poisoning attacks in classification models, and it finds that three predictive criteria—ergodic prediction accuracy, poison distance, and poison budget—effectively quantify this difficulty across diverse scenarios.
Targeted data poisoning attacks pose an increasingly serious threat due to their ease of deployment and high success rates. These attacks aim to manipulate the prediction for a single test sample in classification models. Unlike indiscriminate attacks that aim to decrease overall test performance, targeted attacks present a unique threat to individual test instances. This threat model raises a fundamental question: what factors make certain test samples more susceptible to successful poisoning than others? We investigate how attack difficulty varies across different test instances and identify key characteristics that influence vulnerability. This paper introduces three predictive criteria for targeted data poisoning difficulty: ergodic prediction accuracy (analyzed through clean training dynamics), poison distance, and poison budget. Our experimental results demonstrate that these metrics effectively predict the varying difficulty of real-world targeted poisoning attacks across diverse scenarios, offering practitioners valuable insights for vulnerability assessment and understanding data poisoning attacks.