BASIS: Breach-Aware Selective Prompt Injection Shielding with Prefill Attention Probes
This work is significant for developers of LLM applications, as it aims to improve the user experience and reliability of LLM-based systems by reducing unnecessary rejections due to prompt injection defenses, especially for robust instructions.
The paper addresses the problem of over-refusal in prompt injection defenses for LLMs, where existing methods reject inputs even if the model could handle the injection safely. They propose BASIS, a defense method that uses Attention Competition Ratio to train two linear probes, achieving near-perfect injection detection while significantly reducing over-refusal on safe attack samples across four tasks and six open-source LLMs.
Prompt injection is a critical security threat in large language model (LLM) applications, where attackers hijack model behavior by embedding malicious instructions in user or external data. Existing detection methods only detect the presence of injection and refuse to respond upon detection, overlooking the fact that for many modern aligned models, well-crafted instructions can resist most injection attacks. This means that the injection robustness varies significantly across instructions and models. This leads to widespread unnecessary over-refusal: inputs containing injections that the model could have handled correctly are rejected incorrectly. To deal with this over-refusal issue, we propose BASIS (Robustness-Aware Prompt Injection Defense). This defense method uses the Attention Competition Ratio ($ρ$) as features to train two sparse linear probes: an existence probe and a breach probe. Both probes make defense decisions through cascaded gating, which does not require additional LLM inference. BASIS comprises three stages: injection existence detection, per-sample breach prediction, and instruction robustness assessment; the online cascade refuses only when the model would actually be compromised and thus avoids over-refusal on robust instructions. Experiments across four tasks and six open-source LLMs show that BASIS maintains near-perfect injection detection while substantially reducing over-refusal on safe attack samples, especially under robust instruction templates.