Multi-Agent Generative Adversarial Imitation Learning
This addresses the challenge of learning policies from expert demonstrations in multi-agent systems, which is incremental as it builds upon existing imitation learning concepts.
The paper tackles the problem of imitation learning in multi-agent settings, where existing methods fail due to multiple equilibria and non-stationarity, by proposing a new framework based on generalized inverse reinforcement learning and a practical actor-critic algorithm that achieves good empirical performance in high-dimensional environments.
Imitation learning algorithms can be used to learn a policy from expert demonstrations without access to a reward signal. However, most existing approaches are not applicable in multi-agent settings due to the existence of multiple (Nash) equilibria and non-stationary environments. We propose a new framework for multi-agent imitation learning for general Markov games, where we build upon a generalized notion of inverse reinforcement learning. We further introduce a practical multi-agent actor-critic algorithm with good empirical performance. Our method can be used to imitate complex behaviors in high-dimensional environments with multiple cooperative or competing agents.