SidConArena: An Environment Evaluating Agents in Open-Ended,Positive-Sum Bargaining Game
Provides a dynamic, mixed-motive evaluation environment for LLM agents, addressing the need for benchmarks beyond static reasoning and zero-sum games.
SidConArena introduces a benchmark for evaluating LLM agents in open-ended, positive-sum bargaining games, finding that frontier models achieve higher economic outcomes but still struggle with resource valuation, passive bargaining, and long-term planning.
Evaluating LLM agents requires dynamic environments that go beyond static reasoning and zero-sum games. Real-world economic interaction is often open-ended and mixed-motive: agents must negotiate, create positive-sum surplus, compete for scarce assets, and plan under delayed returns. We introduce SidConArena, a new benchmark framework for evaluating LLM agents in open-ended, positive-sum bargaining. SidConArena formalizes a multi-player economy as a finite-horizon partially observable stochastic game with three coupled phases: natural-language negotiation with binding trades, deterministic converter-based production, and sealed-bid auctions for long-term assets. The framework combines structured observations, phase-aware agent dispatching, a neural-symbolic action interface, and asynchronous execution, enabling free-form interaction while preserving rule-grounded evaluation. Across homogeneous and heterogeneous tournaments, stronger frontier models achieve higher economic outcomes, yet agents still misvalue resources, bargain passively, and remain limited in long-horizon investment planning.