Human-Level Performance in No-Press Diplomacy via Equilibrium Search
This work addresses the problem of developing AI for complex mixed-motive games like Diplomacy, which has been a formidable research challenge due to its shifting alliances, representing a significant advancement beyond purely adversarial or cooperative settings.
The paper tackled the challenge of creating an AI agent for the no-press variant of Diplomacy, a game involving both cooperation and competition, by combining supervised learning on human data with one-step lookahead search via regret minimization, resulting in an agent that greatly exceeds past bots, is unexploitable by expert humans, and ranks in the top 2% of human players in anonymous online games.
Prior AI breakthroughs in complex games have focused on either the purely adversarial or purely cooperative settings. In contrast, Diplomacy is a game of shifting alliances that involves both cooperation and competition. For this reason, Diplomacy has proven to be a formidable research challenge. In this paper we describe an agent for the no-press variant of Diplomacy that combines supervised learning on human data with one-step lookahead search via regret minimization. Regret minimization techniques have been behind previous AI successes in adversarial games, most notably poker, but have not previously been shown to be successful in large-scale games involving cooperation. We show that our agent greatly exceeds the performance of past no-press Diplomacy bots, is unexploitable by expert humans, and ranks in the top 2% of human players when playing anonymous games on a popular Diplomacy website.