Learning Visual Robotic Control Efficiently with Contrastive Pre-training and Data Augmentation
This work is significant for roboticists and AI researchers working on real-world robot learning, as it offers a method to achieve sample-efficient training of robotic arm policies from pixels with sparse rewards, addressing a key bottleneck in deploying RL on physical systems.
This paper addresses the challenge of data-efficient real-robot learning from pixels, where previous unsupervised representation learning gains in simulation have not translated. The authors introduce CoDER, a method combining contrastive pre-training and data augmentation, enabling a single robotic arm to learn sparse-reward manipulation policies from pixels in just 30 minutes of real-world training time, given 10 demonstrations.
Recent advances in unsupervised representation learning significantly improved the sample efficiency of training Reinforcement Learning policies in simulated environments. However, similar gains have not yet been seen for real-robot reinforcement learning. In this work, we focus on enabling data-efficient real-robot learning from pixels. We present Contrastive Pre-training and Data Augmentation for Efficient Robotic Learning (CoDER), a method that utilizes data augmentation and unsupervised learning to achieve sample-efficient training of real-robot arm policies from sparse rewards. While contrastive pre-training, data augmentation, demonstrations, and reinforcement learning are alone insufficient for efficient learning, our main contribution is showing that the combination of these disparate techniques results in a simple yet data-efficient method. We show that, given only 10 demonstrations, a single robotic arm can learn sparse-reward manipulation policies from pixels, such as reaching, picking, moving, pulling a large object, flipping a switch, and opening a drawer in just 30 minutes of mean real-world training time. We include videos and code on the project website: https://sites.google.com/view/efficient-robotic-manipulation/home