No Press Diplomacy: Modeling Multi-Agent Gameplay
This work addresses the challenge of multi-agent gameplay in complex social dilemmas for AI research, though it is incremental as it builds on existing methods for a specific game variant.
The authors tackled the problem of training an agent for the No Press version of Diplomacy, a complex multi-agent game without communication, by developing DipNet, a neural-network-based model trained on over 150,000 human games using supervised and reinforcement learning, which achieved state-of-the-art performance by beating rule-based bots.
Diplomacy is a seven-player non-stochastic, non-cooperative game, where agents acquire resources through a mix of teamwork and betrayal. Reliance on trust and coordination makes Diplomacy the first non-cooperative multi-agent benchmark for complex sequential social dilemmas in a rich environment. In this work, we focus on training an agent that learns to play the No Press version of Diplomacy where there is no dedicated communication channel between players. We present DipNet, a neural-network-based policy model for No Press Diplomacy. The model was trained on a new dataset of more than 150,000 human games. Our model is trained by supervised learning (SL) from expert trajectories, which is then used to initialize a reinforcement learning (RL) agent trained through self-play. Both the SL and RL agents demonstrate state-of-the-art No Press performance by beating popular rule-based bots.