Adversarial Inverse Reinforcement Learning for Mean Field Games
This addresses uncertainties in large-scale multi-agent behavior prediction, but it is incremental as it builds on existing IRL and equilibrium concepts.
The paper tackles the problem of existing inverse reinforcement learning methods for mean field games being unable to handle uncertainties in demonstrated behaviors by proposing MF-AIRL, which shows superior reward recovery in simulated tasks with imperfect demonstrations.
Mean field games (MFGs) provide a mathematically tractable framework for modelling large-scale multi-agent systems by leveraging mean field theory to simplify interactions among agents. It enables applying inverse reinforcement learning (IRL) to predict behaviours of large populations by recovering reward signals from demonstrated behaviours. However, existing IRL methods for MFGs are powerless to reason about uncertainties in demonstrated behaviours of individual agents. This paper proposes a novel framework, Mean-Field Adversarial IRL (MF-AIRL), which is capable of tackling uncertainties in demonstrations. We build MF-AIRL upon maximum entropy IRL and a new equilibrium concept. We evaluate our approach on simulated tasks with imperfect demonstrations. Experimental results demonstrate the superiority of MF-AIRL over existing methods in reward recovery.