A Sober Look at Agentic Misalignment in Automated Workflows
For practitioners building multi-agent automated workflows, this work provides a formal framework and a practical alignment method to reduce emergent misalignment, though results are demonstrated on specific instantiations.
The paper defines and analyzes agentic misalignment in multi-agent systems, where agents follow proxy utilities misaligned with human goals, and proposes Agentic Evidence Attribution (AEA) to correct this via context-specific evidence. Experiments show AEA effectively improves agent collaboration and reliability.
We study a class of emergent misalignment in multi-agent systems (MAS), with a focus on automated workflows, which we refer to agentic misalignment. Although these systems can solve complex tasks, they often fail because agents act according to implicit proxy utilities that do not align with the intended human goals. We formally define these behaviors and analyze them within a Bayesian framework, showing that generic utilities naturally lead to posterior collapse of agents in automated workflows. To address this issue, we propose Agentic Evidence Attribution (AEA), a novel alignment paradigm that improves agent posteriors using context-specific evidence. AEA reasons over agent actions and provides structured evidence to correct misaligned behavior during collaboration. To better understand the role of evidence, we study two instantiations of AEA: self-reflection (internal evidence from the model) and weak-to-strong generalization (external evidence on the agentic trajectory). We show that a small evidence model effectively aligns the MAS by providing orthogonal failure attribution. Our results clarify the sources of agentic misalignment in automated workflows and show that evidence-based alignment can effectively improve agent collaboration and leads to reliable multi-agent systems built on automated workflows.