Cooperative-Competitive Team Play of Real-World Craft Robots
This work addresses the challenge of deploying learned policies on physical robots, which is crucial for advancing robotics applications, though it is incremental in improving existing sim-to-real techniques.
The paper tackles the problem of efficiently training multi-agent reinforcement learning policies for real-world robots and transferring them from simulation to reality, achieving a 20% improvement in sim-to-real performance with their proposed method.
Multi-agent deep Reinforcement Learning (RL) has made significant progress in developing intelligent game-playing agents in recent years. However, the efficient training of collective robots using multi-agent RL and the transfer of learned policies to real-world applications remain open research questions. In this work, we first develop a comprehensive robotic system, including simulation, distributed learning framework, and physical robot components. We then propose and evaluate reinforcement learning techniques designed for efficient training of cooperative and competitive policies on this platform. To address the challenges of multi-agent sim-to-real transfer, we introduce Out of Distribution State Initialization (OODSI) to mitigate the impact of the sim-to-real gap. In the experiments, OODSI improves the Sim2Real performance by 20%. We demonstrate the effectiveness of our approach through experiments with a multi-robot car competitive game and a cooperative task in real-world settings.