Pick Your Battles: Interaction Graphs as Population-Level Objectives for Strategic Diversity
This addresses the problem of strategic diversity in multi-agent systems for game AI, with incremental improvements over existing methods.
The paper tackles the challenge of training diverse populations of agents in games by using interaction graphs to structure agent interactions during training, which leads to improved overall performance as evidenced by analysis across a range of games.
Strategic diversity is often essential in games: in multi-player games, for example, evaluating a player against a diverse set of strategies will yield a more accurate estimate of its performance. Furthermore, in games with non-transitivities diversity allows a player to cover several winning strategies. However, despite the significance of strategic diversity, training agents that exhibit diverse behaviour remains a challenge. In this paper we study how to construct diverse populations of agents by carefully structuring how individuals within a population interact. Our approach is based on interaction graphs, which control the flow of information between agents during training and can encourage agents to specialise on different strategies, leading to improved overall performance. We provide evidence for the importance of diversity in multi-agent training and analyse the effect of applying different interaction graphs on the training trajectories, diversity and performance of populations in a range of games. This is an extended version of the long abstract published at AAMAS.