Enhancing LLM Agent Safety via Causal Influence Prompting
This addresses safety concerns for users of LLM-powered autonomous agents, but it is incremental as it builds on existing causal methods.
The paper tackles the problem of ensuring safe behavior in autonomous LLM agents by introducing CIP, a technique using causal influence diagrams to identify and mitigate risks, and shows it effectively enhances safety in code execution and mobile device control tasks.
As autonomous agents powered by large language models (LLMs) continue to demonstrate potential across various assistive tasks, ensuring their safe and reliable behavior is crucial for preventing unintended consequences. In this work, we introduce CIP, a novel technique that leverages causal influence diagrams (CIDs) to identify and mitigate risks arising from agent decision-making. CIDs provide a structured representation of cause-and-effect relationships, enabling agents to anticipate harmful outcomes and make safer decisions. Our approach consists of three key steps: (1) initializing a CID based on task specifications to outline the decision-making process, (2) guiding agent interactions with the environment using the CID, and (3) iteratively refining the CID based on observed behaviors and outcomes. Experimental results demonstrate that our method effectively enhances safety in both code execution and mobile device control tasks.