RobotKeyframing: Learning Locomotion with High-Level Objectives via Mixture of Dense and Sparse Rewards
This addresses the challenge of incorporating flexible, high-level objectives into robot locomotion control, which is incremental as it builds on existing reinforcement learning methods with specific architectural improvements.
The paper tackles the problem of enabling legged robots to achieve natural locomotion with high-level objectives specified as variable keyframe sequences, using a multi-critic reinforcement learning algorithm and transformer-based encoder, resulting in effective target satisfaction and reduced hyperparameter tuning effort.
This paper presents a novel learning-based control framework that uses keyframing to incorporate high-level objectives in natural locomotion for legged robots. These high-level objectives are specified as a variable number of partial or complete pose targets that are spaced arbitrarily in time. Our proposed framework utilizes a multi-critic reinforcement learning algorithm to effectively handle the mixture of dense and sparse rewards. Additionally, it employs a transformer-based encoder to accommodate a variable number of input targets, each associated with specific time-to-arrivals. Throughout simulation and hardware experiments, we demonstrate that our framework can effectively satisfy the target keyframe sequence at the required times. In the experiments, the multi-critic method significantly reduces the effort of hyperparameter tuning compared to the standard single-critic alternative. Moreover, the proposed transformer-based architecture enables robots to anticipate future goals, which results in quantitative improvements in their ability to reach their targets.