Reinforcement Learning Enabled Adaptive Multi-Task Control for Bipedal Soccer Robots
For bipedal soccer robots, this work addresses the challenge of motion stability and multi-task control switching, enabling seamless operation in dynamic environments.
This paper proposes a modular reinforcement learning framework for bipedal soccer robots that combines an open-loop oscillator with RL-based feedback residuals and a posture-driven state machine for multi-task control. The system achieves reliable ball seeking and kicking in restricted corner scenarios and rapid fall recovery with an average recovery time of 0.715 seconds.
Developing bipedal football robots in dynamiccombat environments presents challenges related to motionstability and deep coupling of multiple tasks, as well ascontrol switching issues between different states such as up-right walking and fall recovery. To address these problems,this paper proposes a modular reinforcement learning (RL)framework for achieving adaptive multi-task control. Firstly,this framework combines an open-loop feedforward oscilla-tor with a reinforcement learning-based feedback residualstrategy, effectively separating the generation of basic gaitsfrom complex football actions. Secondly, a posture-driven statemachine is introduced, clearly switching between the ballseeking and kicking network (BSKN) and the fall recoverynetwork (FRN), fundamentally preventing state interference.The FRN is efficiently trained through a progressive forceattenuation curriculum learning strategy. The architecture wasverified in Unity simulations of bipedal robots, demonstratingexcellent spatial adaptability-reliably finding and kicking theball even in restricted corner scenarios-and rapid autonomousfall recovery (with an average recovery time of 0.715 seconds).This ensures seamless and stable operation in complex multi-task environments.