Interpreting Emergent Extreme Events in Multi-Agent Systems
This addresses the critical need for system safety in multi-agent systems by providing interpretability for extreme events, though it is an incremental advance in attribution methods.
The paper tackles the problem of interpreting emergent extreme events in multi-agent systems by proposing a framework that uses Shapley values to attribute events to actions, agents, and behaviors, demonstrating effectiveness across economic, financial, and social scenarios.
Large language model-powered multi-agent systems have emerged as powerful tools for simulating complex human-like systems. The interactions within these systems often lead to extreme events whose origins remain obscured by the black box of emergence. Interpreting these events is critical for system safety. This paper proposes the first framework for explaining emergent extreme events in multi-agent systems, aiming to answer three fundamental questions: When does the event originate? Who drives it? And what behaviors contribute to it? Specifically, we adapt the Shapley value to faithfully attribute the occurrence of extreme events to each action taken by agents at different time steps, i.e., assigning an attribution score to the action to measure its influence on the event. We then aggregate the attribution scores along the dimensions of time, agent, and behavior to quantify the risk contribution of each dimension. Finally, we design a set of metrics based on these contribution scores to characterize the features of extreme events. Experiments across diverse multi-agent system scenarios (economic, financial, and social) demonstrate the effectiveness of our framework and provide general insights into the emergence of extreme phenomena.