Toward Real-World Cooperative and Competitive Soccer with Quadrupedal Robot Teams
This work addresses the problem of achieving coordinated teamwork in dynamic, competitive multi-agent environments for legged robots, representing an incremental advancement in applying MARL to real-world robotic soccer.
The paper tackled the challenge of enabling fully autonomous and decentralized quadruped robot soccer by developing a hierarchical multi-agent reinforcement learning framework, resulting in a system that supports autonomous robot-robot and robot-human matches on indoor and outdoor courts with significant advantages in cooperative and competitive gameplay.
Achieving coordinated teamwork among legged robots requires both fine-grained locomotion control and long-horizon strategic decision-making. Robot soccer offers a compelling testbed for this challenge, combining dynamic, competitive, and multi-agent interactions. In this work, we present a hierarchical multi-agent reinforcement learning (MARL) framework that enables fully autonomous and decentralized quadruped robot soccer. First, a set of highly dynamic low-level skills is trained for legged locomotion and ball manipulation, such as walking, dribbling, and kicking. On top of these, a high-level strategic planning policy is trained with Multi-Agent Proximal Policy Optimization (MAPPO) via Fictitious Self-Play (FSP). This learning framework allows agents to adapt to diverse opponent strategies and gives rise to sophisticated team behaviors, including coordinated passing, interception, and dynamic role allocation. With an extensive ablation study, the proposed learning method shows significant advantages in the cooperative and competitive multi-agent soccer game. We deploy the learned policies to real quadruped robots relying solely on onboard proprioception and decentralized localization, with the resulting system supporting autonomous robot-robot and robot-human soccer matches on indoor and outdoor soccer courts.