Comparison of Multi-agent and Single-agent Inverse Learning on a Simulated Soccer Example
This work addresses the problem of reward inference in multi-agent systems for researchers, but it is incremental as it extends an existing method and compares it in a specific domain.
The authors compared single-agent Inverse Reinforcement Learning (IRL) with Multi-agent IRL (MIRL) on a simulated soccer game, finding that IRL performed much worse than MIRL, likely due to its inability to capture equilibrium information.
We compare the performance of Inverse Reinforcement Learning (IRL) with the relative new model of Multi-agent Inverse Reinforcement Learning (MIRL). Before comparing the methods, we extend a published Bayesian IRL approach that is only applicable to the case where the reward is only state dependent to a general one capable of tackling the case where the reward depends on both state and action. Comparison between IRL and MIRL is made in the context of an abstract soccer game, using both a game model in which the reward depends only on state and one in which it depends on both state and action. Results suggest that the IRL approach performs much worse than the MIRL approach. We speculate that the underperformance of IRL is because it fails to capture equilibrium information in the manner possible in MIRL.