Multi-Agent Systems are Mixtures of Experts: Who Becomes an Influencer?
Provides a theoretical framework for understanding and improving multi-agent LLM systems, relevant to researchers designing collaborative AI systems.
The paper models multi-agent LLM deliberation using Friedkin-Johnsen opinion dynamics, showing it acts as a mixture of experts where input-dependent parameters enable outperformance of single agents and static ensembles when routing reflects competence. It analyzes how influence is established via confidence and alignment proxies.
The effectiveness of multi-agent LLM deliberation depends not only on the agents' individual predictions, but also on how they communicate and collaborate. We study this mechanism through the lens of Friedkin-Johnsen (FJ) opinion dynamics, a tractable model for analyzing stubbornness, influence, and opinion change in multi-agent systems that captures empirically observed deliberation patterns. We show that the FJ parameters are input-dependent, turning multi-agent deliberation into a mixture of experts. This perspective implies that multi-agent systems can outperform single agents and static ensembles when routing reflects agent competence. Since competence is latent in practice, we analyze how influence is established through observable proxies: agents' self-assessed confidence, their perceived confidence, and initial alignment with other agents' views.