Recursive language models for jailbreak detection: a procedural defense for tool-augmented agents
This addresses a practical and evolving security threat for users of LLM-based agents, offering a procedural defense against jailbreak attacks.
The paper tackles the problem of jailbreak prompts in large language models, particularly in agentic systems, by introducing RLM-JB, a recursive language model framework that achieves high detection effectiveness (ASR/Recall 92.5-98.0%) and precision (98.99-100%) with low false positive rates (0.0-2.0%).
Jailbreak prompts are a practical and evolving threat to large language models (LLMs), particularly in agentic systems that execute tools over untrusted content. Many attacks exploit long-context hiding, semantic camouflage, and lightweight obfuscations that can evade single-pass guardrails. We present RLM-JB, an end-to-end jailbreak detection framework built on Recursive Language Models (RLMs), in which a root model orchestrates a bounded analysis program that transforms the input, queries worker models over covered segments, and aggregates evidence into an auditable decision. RLM-JB treats detection as a procedure rather than a one-shot classification: it normalizes and de-obfuscates suspicious inputs, chunks text to reduce context dilution and guarantee coverage, performs parallel chunk screening, and composes cross-chunk signals to recover split-payload attacks. On AutoDAN-style adversarial inputs, RLM-JB achieves high detection effectiveness across three LLM backends (ASR/Recall 92.5-98.0%) while maintaining very high precision (98.99-100%) and low false positive rates (0.0-2.0%), highlighting a practical sensitivity-specificity trade-off as the screening backend changes.