Threats to Pre-trained Language Models: Survey and Taxonomy
It addresses security concerns for researchers and practitioners using PTLMs, but is incremental as it systematizes existing knowledge rather than introducing new methods.
The paper surveys and taxonomizes security threats to pre-trained language models (PTLMs), categorizing attacks by pipeline stages, transferability types, and goals such as backdoor and privacy breaches.
Pre-trained language models (PTLMs) have achieved great success and remarkable performance over a wide range of natural language processing (NLP) tasks. However, there are also growing concerns regarding the potential security issues in the adoption of PTLMs. In this survey, we comprehensively systematize recently discovered threats to PTLM systems and applications. We perform our attack characterization from three interesting perspectives. (1) We show threats can occur at different stages of the PTLM pipeline raised by different malicious entities. (2) We identify two types of model transferability (landscape, portrait) that facilitate attacks. (3) Based on the attack goals, we summarize four categories of attacks (backdoor, evasion, data privacy and model privacy). We also discuss some open problems and research directions. We believe our survey and taxonomy will inspire future studies towards secure and privacy-preserving PTLMs.