BadAgent: Inserting and Activating Backdoor Attacks in LLM Agents
This work highlights a critical security risk for users deploying LLM agents, as it exposes how untrusted models or data can lead to dangerous tool misuse, representing a novel domain-specific threat rather than an incremental advance.
The paper tackles the vulnerability of LLM-based agents to backdoor attacks, showing that fine-tuning on backdoor data can embed triggers that cause agents to execute harmful operations, with attacks remaining robust even after subsequent fine-tuning on trustworthy data.
With the prosperity of large language models (LLMs), powerful LLM-based intelligent agents have been developed to provide customized services with a set of user-defined tools. State-of-the-art methods for constructing LLM agents adopt trained LLMs and further fine-tune them on data for the agent task. However, we show that such methods are vulnerable to our proposed backdoor attacks named BadAgent on various agent tasks, where a backdoor can be embedded by fine-tuning on the backdoor data. At test time, the attacker can manipulate the deployed LLM agents to execute harmful operations by showing the trigger in the agent input or environment. To our surprise, our proposed attack methods are extremely robust even after fine-tuning on trustworthy data. Though backdoor attacks have been studied extensively in natural language processing, to the best of our knowledge, we could be the first to study them on LLM agents that are more dangerous due to the permission to use external tools. Our work demonstrates the clear risk of constructing LLM agents based on untrusted LLMs or data. Our code is public at https://github.com/DPamK/BadAgent