Security--Fidelity Tradeoffs: The Hidden Cost of Prompt Injection Defense
For researchers and practitioners deploying LLMs in security-sensitive applications, this work reveals that reporting only attack-success metrics hides the cost of suppressed benign content, making fidelity a necessary complementary metric.
The paper identifies a security-fidelity tradeoff in defending LLMs against indirect prompt injection: defenses that resist injected instructions often suppress untrusted text, which corrupts tasks like translation and document editing. The authors introduce SecFid, a benchmark showing that no model or defense achieves both objectives, with the highest-fidelity model reaching 96.5% fidelity at 47.8% security, while the most secure defenses achieve 99.3% security but only 71.0%-73.9% fidelity.
We identify a security-fidelity tradeoff in defending LLMs against indirect prompt injection: defenses resist injected instructions largely by suppressing untrusted text, which corrupts tasks that must preserve it, such as translation and document editing. Attack-success metrics cannot see this, because a model that ignores an injection and one that faithfully processes it as data score identically. We introduce SecFid, a benchmark built so that executing an injection, processing it as data, and ignoring it produce distinguishable outputs. This makes fidelity measurable and exposes a frontier: across 1,168 examples and 48 configurations, no model or defense achieves both objectives. The highest-fidelity model reaches 96.5% fidelity at 47.8% security, while the most secure defenses invert this, at 99.3% security but only 71.0%-73.9% fidelity. Even defenses with identical security differ in how they earn it: some repair hijacks into faithful processing, others simply suppress benign content. A decision-theoretic analysis shows why no fixed choice can be right everywhere: the correct behavior is not a property of the defense but of the deployment, set by its relative cost of a hijack versus a dropped span. Security alone therefore measures only half of robustness, and reporting it without fidelity hides the price at which it was bought.