Real-World Humanoid Locomotion with Reinforcement Learning
This addresses the problem of labor shortages and assistance needs by enabling more adaptable humanoid robots, though it builds incrementally on existing learning-based methods.
The paper tackled the challenge of generalizing humanoid locomotion to diverse real-world environments by developing a fully learning-based controller using a causal transformer trained with model-free reinforcement learning in simulation, achieving zero-shot deployment and robust performance on various outdoor terrains with adaptation to disturbances.
Humanoid robots that can autonomously operate in diverse environments have the potential to help address labour shortages in factories, assist elderly at homes, and colonize new planets. While classical controllers for humanoid robots have shown impressive results in a number of settings, they are challenging to generalize and adapt to new environments. Here, we present a fully learning-based approach for real-world humanoid locomotion. Our controller is a causal transformer that takes the history of proprioceptive observations and actions as input and predicts the next action. We hypothesize that the observation-action history contains useful information about the world that a powerful transformer model can use to adapt its behavior in-context, without updating its weights. We train our model with large-scale model-free reinforcement learning on an ensemble of randomized environments in simulation and deploy it to the real world zero-shot. Our controller can walk over various outdoor terrains, is robust to external disturbances, and can adapt in context.