Stop Fixating on Prompts: Reasoning Hijacking and Constraint Tightening for Red-Teaming LLM Agents
This addresses security threats in LLM agents for developers and users, though it appears incremental as an improved red-teaming method.
The paper tackles the security vulnerability of LLM-based agents to prompt-based attacks by proposing JailAgent, a framework that manipulates reasoning trajectories and memory retrieval without modifying user prompts, achieving strong performance in cross-model and cross-scenario tests.
With the widespread application of LLM-based agents across various domains, their complexity has introduced new security threats. Existing red-team methods mostly rely on modifying user prompts, which lack adaptability to new data and may impact the agent's performance. To address the challenge, this paper proposes the JailAgent framework, which completely avoids modifying the user prompt. Specifically, it implicitly manipulates the agent's reasoning trajectory and memory retrieval with three key stages: Trigger Extraction, Reasoning Hijacking, and Constraint Tightening. Through precise trigger identification, real-time adaptive mechanisms, and an optimized objective function, JailAgent demonstrates outstanding performance in cross-model and cross-scenario environments.