DemoStart: Demonstration-led auto-curriculum applied to sim-to-real with multi-fingered robots
This work addresses the challenge of efficient robot behavior generation for manipulation tasks, offering a significant reduction in demonstration requirements and enabling sim-to-real transfer, though it is incremental in combining existing techniques like auto-curriculum and domain randomization.
The paper tackles the problem of learning complex manipulation behaviors for multi-fingered robots by introducing DemoStart, an auto-curriculum reinforcement learning method that uses sparse rewards and a few demonstrations in simulation, achieving zero-shot sim-to-real transfer and outperforming real-robot demonstration policies while requiring 100 times fewer demonstrations.
We present DemoStart, a novel auto-curriculum reinforcement learning method capable of learning complex manipulation behaviors on an arm equipped with a three-fingered robotic hand, from only a sparse reward and a handful of demonstrations in simulation. Learning from simulation drastically reduces the development cycle of behavior generation, and domain randomization techniques are leveraged to achieve successful zero-shot sim-to-real transfer. Transferred policies are learned directly from raw pixels from multiple cameras and robot proprioception. Our approach outperforms policies learned from demonstrations on the real robot and requires 100 times fewer demonstrations, collected in simulation. More details and videos in https://sites.google.com/view/demostart.