Learning Agile Locomotion Skills with a Mentor
This work addresses the problem of tedious reward shaping in deep reinforcement learning for agile robot locomotion, offering an incremental improvement for robotics researchers.
The paper tackles the challenge of developing agile locomotion for legged robots by introducing a multi-stage learning approach with a mentor to guide training, resulting in a simulated quadruped robot that significantly outperforms a baseline in running and jumping across gaps and hurdles.
Developing agile behaviors for legged robots remains a challenging problem. While deep reinforcement learning is a promising approach, learning truly agile behaviors typically requires tedious reward shaping and careful curriculum design. We formulate agile locomotion as a multi-stage learning problem in which a mentor guides the agent throughout the training. The mentor is optimized to place a checkpoint to guide the movement of the robot's center of mass while the student (i.e. the robot) learns to reach these checkpoints. Once the student can solve the task, we teach the student to perform the task without the mentor. We evaluate our proposed learning system with a simulated quadruped robot on a course consisting of randomly generated gaps and hurdles. Our method significantly outperforms a single-stage RL baseline without a mentor, and the quadruped robot can agilely run and jump across gaps and obstacles. Finally, we present a detailed analysis of the learned behaviors' feasibility and efficiency.