RACER: Rich Language-Guided Failure Recovery Policies for Imitation Learning
This work addresses the problem of robot failure recovery in manipulation tasks for robotics, offering a scalable solution with improved performance, though it appears incremental as it builds on existing imitation learning and language-guided methods.
The paper tackles the challenge of developing robust and correctable visuomotor policies for robotic manipulation by proposing RACER, a supervisor-actor framework that uses rich language guidance for failure recovery, which outperforms the state-of-the-art RVT on RLbench across various tasks in simulated and real-world environments.
Developing robust and correctable visuomotor policies for robotic manipulation is challenging due to the lack of self-recovery mechanisms from failures and the limitations of simple language instructions in guiding robot actions. To address these issues, we propose a scalable data generation pipeline that automatically augments expert demonstrations with failure recovery trajectories and fine-grained language annotations for training. We then introduce Rich languAge-guided failure reCovERy (RACER), a supervisor-actor framework, which combines failure recovery data with rich language descriptions to enhance robot control. RACER features a vision-language model (VLM) that acts as an online supervisor, providing detailed language guidance for error correction and task execution, and a language-conditioned visuomotor policy as an actor to predict the next actions. Our experimental results show that RACER outperforms the state-of-the-art Robotic View Transformer (RVT) on RLbench across various evaluation settings, including standard long-horizon tasks, dynamic goal-change tasks and zero-shot unseen tasks, achieving superior performance in both simulated and real world environments. Videos and code are available at: https://rich-language-failure-recovery.github.io.