Recognize Your Orchestrator: An Entropy Dynamics Perspective for LLM Multi-Agent Systems
For researchers designing LLM multi-agent systems, this work provides a physically interpretable framework to quantify system stability and performance collapse, highlighting a critical failure mode of reasoning-heavy orchestrators.
The paper proposes a Mean-Field Entropy Dynamics framework to model centralized orchestration in LLM multi-agent systems, and introduces Inverse Workflow Generation (IWG) for benchmarking. The model fits empirical trajectories, revealing a 'Reasoning Trap' where reasoning-heavy models fail as orchestrators due to context squeezing.
The transition from single-turn models to Multi-Agent Systems (MAS) promises enhanced problem-solving capabilities, yet the centralized orchestration topology remains a critical point of fragility. To analyze this, we propose a Mean-Field Entropy Dynamics framework, modeling the orchestration process as a system governed by the competing forces of task resolution and cumulative context loading. To facilitate validation, we introduce Inverse Workflow Generation (IWG), a multi-agent pipeline that synthesizes process-verifiable, high-complexity benchmarks with dense intermediate checkpoints. We demonstrate that our entropy dynamics model fits empirical trajectories, providing physically interpretable parameters that quantify system stability and performance collapse. Crucially, our analysis uncovers a ``Reasoning Trap": while reasoning-heavy models excel in isolated tasks, they frequently fail as orchestrators due to context squeezing. Elucidating the physical mechanisms underlying the Orchestrator and quantifying systemic uncertainty offers insights for the MASs' architectural design.