Truthful Self-Play
This addresses the issue of information asymmetry in multi-agent systems for researchers in AI and reinforcement learning, though it appears incremental as it builds on existing self-play and mechanism design concepts.
The paper tackles the problem of evolutionary learning converging to bad local optima in multi-agent reinforcement learning in non-cooperative partially observable environments with communication, by proposing a framework that modifies self-play with mechanism design to elicit truthful signals and make agents cooperative, achieving state-of-the-art performance in tasks like predator prey, traffic junction, and StarCraft.
We present a general framework for evolutionary learning to emergent unbiased state representation without any supervision. Evolutionary frameworks such as self-play converge to bad local optima in case of multi-agent reinforcement learning in non-cooperative partially observable environments with communication due to information asymmetry. Our proposed framework is a simple modification of self-play inspired by mechanism design, also known as {\em reverse game theory}, to elicit truthful signals and make the agents cooperative. The key idea is to add imaginary rewards using the peer prediction method, i.e., a mechanism for evaluating the validity of information exchanged between agents in a decentralized environment. Numerical experiments with predator prey, traffic junction and StarCraft tasks demonstrate that the state-of-the-art performance of our framework.