Coordinated Multi-Agent Imitation Learning
This addresses the challenge of learning coordinated multi-agent policies from demonstrations, which is incremental as it builds on existing imitation learning methods.
The paper tackles the problem of imitation learning from demonstrations of multiple coordinating agents by proposing a joint approach that simultaneously learns a latent coordination model and individual policies. The method shows substantially improved imitation loss compared to conventional baselines on a team sports behavior modeling task.
We study the problem of imitation learning from demonstrations of multiple coordinating agents. One key challenge in this setting is that learning a good model of coordination can be difficult, since coordination is often implicit in the demonstrations and must be inferred as a latent variable. We propose a joint approach that simultaneously learns a latent coordination model along with the individual policies. In particular, our method integrates unsupervised structure learning with conventional imitation learning. We illustrate the power of our approach on a difficult problem of learning multiple policies for fine-grained behavior modeling in team sports, where different players occupy different roles in the coordinated team strategy. We show that having a coordination model to infer the roles of players yields substantially improved imitation loss compared to conventional baselines.