Symmetry-Guided Multi-Agent Inverse Reinforcement Learning
This addresses the critical challenge of sample efficiency for deploying multi-agent inverse reinforcement learning in practical robotic applications, though it appears incremental as it builds on existing algorithms.
The paper tackles the problem of high sample requirements in multi-agent inverse reinforcement learning by proposing a symmetry-guided framework that integrates symmetry into existing algorithms, which experimental results show significantly enhances sample efficiency and has been validated in physical multi-robot systems.
In robotic systems, the performance of reinforcement learning depends on the rationality of predefined reward functions. However, manually designed reward functions often lead to policy failures due to inaccuracies. Inverse Reinforcement Learning (IRL) addresses this problem by inferring implicit reward functions from expert demonstrations. Nevertheless, existing methods rely heavily on large amounts of expert demonstrations to accurately recover the reward function. The high cost of collecting expert demonstrations in robotic applications, particularly in multi-robot systems, severely hinders the practical deployment of IRL. Consequently, improving sample efficiency has emerged as a critical challenge in multi-agent inverse reinforcement learning (MIRL). Inspired by the symmetry inherent in multi-agent systems, this work theoretically demonstrates that leveraging symmetry enables the recovery of more accurate reward functions. Building upon this insight, we propose a universal framework that integrates symmetry into existing multi-agent adversarial IRL algorithms, thereby significantly enhancing sample efficiency. Experimental results from multiple challenging tasks have demonstrated the effectiveness of this framework. Further validation in physical multi-robot systems has shown the practicality of our method.