"Other-Play" for Zero-Shot Coordination
This addresses the challenge of creating AI agents that can coordinate effectively with novel partners like humans, which is crucial for real-world applications, though it is an incremental improvement over existing methods.
The paper tackles the problem of zero-shot coordination in AI agents by introducing the other-play algorithm, which enhances self-play to find more robust strategies using known symmetries, and shows that OP agents achieve higher scores with independently trained agents and human players compared to state-of-the-art self-play agents.
We consider the problem of zero-shot coordination - constructing AI agents that can coordinate with novel partners they have not seen before (e.g. humans). Standard Multi-Agent Reinforcement Learning (MARL) methods typically focus on the self-play (SP) setting where agents construct strategies by playing the game with themselves repeatedly. Unfortunately, applying SP naively to the zero-shot coordination problem can produce agents that establish highly specialized conventions that do not carry over to novel partners they have not been trained with. We introduce a novel learning algorithm called other-play (OP), that enhances self-play by looking for more robust strategies, exploiting the presence of known symmetries in the underlying problem. We characterize OP theoretically as well as experimentally. We study the cooperative card game Hanabi and show that OP agents achieve higher scores when paired with independently trained agents. In preliminary results we also show that our OP agents obtains higher average scores when paired with human players, compared to state-of-the-art SP agents.