Thieves on Sesame Street! Model Extraction of BERT-based APIs
This exposes a security vulnerability in NLP transfer learning systems, posing a threat to model owners and users, and is incremental in highlighting an exploit enabled by current practices.
The authors tackled model extraction attacks on BERT-based APIs, showing that attackers can reconstruct a local copy using random word sequences and task-specific heuristics without real training data, achieving performance only slightly worse than the victim model for a query budget of a few hundred dollars.
We study the problem of model extraction in natural language processing, in which an adversary with only query access to a victim model attempts to reconstruct a local copy of that model. Assuming that both the adversary and victim model fine-tune a large pretrained language model such as BERT (Devlin et al. 2019), we show that the adversary does not need any real training data to successfully mount the attack. In fact, the attacker need not even use grammatical or semantically meaningful queries: we show that random sequences of words coupled with task-specific heuristics form effective queries for model extraction on a diverse set of NLP tasks, including natural language inference and question answering. Our work thus highlights an exploit only made feasible by the shift towards transfer learning methods within the NLP community: for a query budget of a few hundred dollars, an attacker can extract a model that performs only slightly worse than the victim model. Finally, we study two defense strategies against model extraction---membership classification and API watermarking---which while successful against naive adversaries, are ineffective against more sophisticated ones.