Efficient but Vulnerable: Benchmarking and Defending LLM Batch Prompting Attack
This addresses a critical security problem for users of efficient LLM inference methods, though it is incremental as it builds on known batch prompting techniques.
The paper tackles the security vulnerability in batch prompting for LLMs, where malicious instructions can cause harmful interference across queries, and finds that all tested LLMs are susceptible, with a probing-based defense achieving about 95% detection accuracy.
Batch prompting, which combines a batch of multiple queries sharing the same context in one inference, has emerged as a promising solution to reduce inference costs. However, our study reveals a significant security vulnerability in batch prompting: malicious users can inject attack instructions into a batch, leading to unwanted interference across all queries, which can result in the inclusion of harmful content, such as phishing links, or the disruption of logical reasoning. In this paper, we construct BATCHSAFEBENCH, a comprehensive benchmark comprising 150 attack instructions of two types and 8k batch instances, to study the batch prompting vulnerability systematically. Our evaluation of both closed-source and open-weight LLMs demonstrates that all LLMs are susceptible to batch-prompting attacks. We then explore multiple defending approaches. While the prompting-based defense shows limited effectiveness for smaller LLMs, the probing-based approach achieves about 95% accuracy in detecting attacks. Additionally, we perform a mechanistic analysis to understand the attack and identify attention heads that are responsible for it.