Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation
This addresses the problem of robot skill acquisition for researchers and practitioners in robotics, but it is incremental as it builds on existing imitation learning methods with new teleoperation hardware.
The paper tackles the challenge of obtaining demonstrations for imitation learning by using consumer-grade Virtual Reality hardware to teleoperate robots for complex tasks, and shows that deep neural network policies can learn these skills from pixels to actions, with experiments demonstrating effectiveness for visuomotor skills.
Imitation learning is a powerful paradigm for robot skill acquisition. However, obtaining demonstrations suitable for learning a policy that maps from raw pixels to actions can be challenging. In this paper we describe how consumer-grade Virtual Reality headsets and hand tracking hardware can be used to naturally teleoperate robots to perform complex tasks. We also describe how imitation learning can learn deep neural network policies (mapping from pixels to actions) that can acquire the demonstrated skills. Our experiments showcase the effectiveness of our approach for learning visuomotor skills.