Beyond Labeling Oracles: What does it mean to steal ML models?
This work addresses a critical oversight in evaluating model extraction attacks for ML-as-a-Service providers, highlighting that current methods misinterpret performance, which is incremental but important for security assessments.
The paper investigates the assumption in model extraction attacks that attackers save on data acquisition and labeling costs, finding that attackers often do not because current attacks rely on access to in-distribution data, with prior knowledge dominating factors like attack policy.
Model extraction attacks are designed to steal trained models with only query access, as is often provided through APIs that ML-as-a-Service providers offer. Machine Learning (ML) models are expensive to train, in part because data is hard to obtain, and a primary incentive for model extraction is to acquire a model while incurring less cost than training from scratch. Literature on model extraction commonly claims or presumes that the attacker is able to save on both data acquisition and labeling costs. We thoroughly evaluate this assumption and find that the attacker often does not. This is because current attacks implicitly rely on the adversary being able to sample from the victim model's data distribution. We thoroughly research factors influencing the success of model extraction. We discover that prior knowledge of the attacker, i.e., access to in-distribution data, dominates other factors like the attack policy the adversary follows to choose which queries to make to the victim model API. Our findings urge the community to redefine the adversarial goals of ME attacks as current evaluation methods misinterpret the ME performance.