Specification-Guided Learning of Nash Equilibria with High Social Welfare
This addresses the challenge of designing efficient and fair multi-agent reinforcement learning policies for applications like robotics or game theory, though it appears incremental as it builds on existing specification and equilibrium concepts.
The paper tackles the problem of training joint policies that form a Nash equilibrium in non-cooperative multi-agent systems, achieving high social welfare, with empirical results showing it outperforms state-of-the-art baselines that either fail or produce lower welfare.
Reinforcement learning has been shown to be an effective strategy for automatically training policies for challenging control problems. Focusing on non-cooperative multi-agent systems, we propose a novel reinforcement learning framework for training joint policies that form a Nash equilibrium. In our approach, rather than providing low-level reward functions, the user provides high-level specifications that encode the objective of each agent. Then, guided by the structure of the specifications, our algorithm searches over policies to identify one that provably forms an $ε$-Nash equilibrium (with high probability). Importantly, it prioritizes policies in a way that maximizes social welfare across all agents. Our empirical evaluation demonstrates that our algorithm computes equilibrium policies with high social welfare, whereas state-of-the-art baselines either fail to compute Nash equilibria or compute ones with comparatively lower social welfare.