SODE: Analyzing Social Dynamics in LLM Agents
Provides a mechanism-grounded benchmark for evaluating and aligning LLM agents with human social dynamics, addressing limitations of outcome-based metrics.
The paper introduces SODE, a framework to evaluate LLM agents on social dynamics dimensions (direct/indirect reciprocity, group dynamics), revealing that instruction-tuned models show passive compliance and reasoning models prioritize short-term gains, but long-horizon framing improves reciprocal behavior.
As Large Language Models (LLMs) evolve into interactive agents, understanding their behavioral alignment within human social dynamics becomes essential. While behavioral game theory offers a framework to study these interactions, previous work has predominantly relied on outcome-based metrics such as average scores. This focus overlooks the mechanisms that facilitate sustainable cooperation, as identical scores can be derived from vastly different strategies. To bridge this gap, we introduce SODE (Social Dynamics Evaluation), a framework that evaluates LLM agents across three evolutionary dimensions: Direct Reciprocity for strategy adaptation, Indirect Reciprocity for reputation sensitivity, and Group Dynamics for cooperative resilience. Applying SODE reveals systematic divergences: instruction-tuned models often exhibit "passive compliance" that renders them vulnerable to exploitation, while reasoning models prioritize short-horizon optimization, destabilizing long-term cooperation. Notably, we demonstrate that a "long-horizon framing" can unlock reciprocal capabilities in reasoning models. Thus, SODE offers a systematic, mechanism-grounded benchmark for aligning AI agents with complex human social dynamics.