Learning Generalizable Locomotion Skills with Hierarchical Reinforcement Learning
This addresses the challenge of efficient and generalizable locomotion for robotics, representing a strong specific gain rather than a broad paradigm shift.
The paper tackles the problem of learning goal-oriented locomotion on real-world robots by introducing a hierarchical reinforcement learning framework that improves sample efficiency and generalizability. The result is that the robot can learn to walk to arbitrary goals up to 12 meters away after about two hours of training from scratch on hardware.
Learning to locomote to arbitrary goals on hardware remains a challenging problem for reinforcement learning. In this paper, we present a hierarchical learning framework that improves sample-efficiency and generalizability of locomotion skills on real-world robots. Our approach divides the problem of goal-oriented locomotion into two sub-problems: learning diverse primitives skills, and using model-based planning to sequence these skills. We parametrize our primitives as cyclic movements, improving sample-efficiency of learning on a 18 degrees of freedom robot. Then, we learn coarse dynamics models over primitive cycles and use them in a model predictive control framework. This allows us to learn to walk to arbitrary goals up to 12m away, after about two hours of training from scratch on hardware. Our results on a Daisy hexapod hardware and simulation demonstrate the efficacy of our approach at reaching distant targets, in different environments and with sensory noise.