Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformers
This work addresses the problem of enabling robots to navigate complex environments more effectively, representing an incremental improvement over existing methods by integrating vision for proactive control.
The authors tackled the problem of quadrupedal locomotion in challenging terrains by introducing LocoTransformer, an end-to-end reinforcement learning method that combines proprioceptive and visual inputs using Transformers, resulting in significantly improved generalization and successful transfer from simulation to real robots with unseen obstacles.
We propose to address quadrupedal locomotion tasks using Reinforcement Learning (RL) with a Transformer-based model that learns to combine proprioceptive information and high-dimensional depth sensor inputs. While learning-based locomotion has made great advances using RL, most methods still rely on domain randomization for training blind agents that generalize to challenging terrains. Our key insight is that proprioceptive states only offer contact measurements for immediate reaction, whereas an agent equipped with visual sensory observations can learn to proactively maneuver environments with obstacles and uneven terrain by anticipating changes in the environment many steps ahead. In this paper, we introduce LocoTransformer, an end-to-end RL method that leverages both proprioceptive states and visual observations for locomotion control. We evaluate our method in challenging simulated environments with different obstacles and uneven terrain. We transfer our learned policy from simulation to a real robot by running it indoors and in the wild with unseen obstacles and terrain. Our method not only significantly improves over baselines, but also achieves far better generalization performance, especially when transferred to the real robot. Our project page with videos is at https://rchalyang.github.io/LocoTransformer/ .