PoisonPrompt: Backdoor Attack on Prompt-based Large Language Models
This work addresses a serious security threat for users of prompt-based LLMs, highlighting potential vulnerabilities in increasingly indispensable applications, and is incremental in exploring an underexplored area of backdoor attacks.
The paper tackles the backdoor vulnerability in prompt-based large language models by introducing POISONPROMPT, a novel attack that successfully compromises both hard and soft prompt-based LLMs, as demonstrated through extensive experiments on three prompt methods, six datasets, and three LLMs.
Prompts have significantly improved the performance of pretrained Large Language Models (LLMs) on various downstream tasks recently, making them increasingly indispensable for a diverse range of LLM application scenarios. However, the backdoor vulnerability, a serious security threat that can maliciously alter the victim model's normal predictions, has not been sufficiently explored for prompt-based LLMs. In this paper, we present POISONPROMPT, a novel backdoor attack capable of successfully compromising both hard and soft prompt-based LLMs. We evaluate the effectiveness, fidelity, and robustness of POISONPROMPT through extensive experiments on three popular prompt methods, using six datasets and three widely used LLMs. Our findings highlight the potential security threats posed by backdoor attacks on prompt-based LLMs and emphasize the need for further research in this area.