Green Screen Augmentation Enables Scene Generalisation in Robotic Manipulation
This addresses the scene generalization limitation in robot learning for researchers and practitioners, though it is incremental as it builds on existing data augmentation techniques without algorithmic novelties.
The paper tackles the problem of generalizing vision-based robotic manipulation policies to new environments by proposing Green-screen Augmentation (GreenAug), which uses chroma keying to overlay background textures on green screen data, and demonstrates through over 850 training demonstrations and 8.2k evaluation episodes that it outperforms no augmentation, standard computer vision augmentations, and prior generative methods.
Generalising vision-based manipulation policies to novel environments remains a challenging area with limited exploration. Current practices involve collecting data in one location, training imitation learning or reinforcement learning policies with this data, and deploying the policy in the same location. However, this approach lacks scalability as it necessitates data collection in multiple locations for each task. This paper proposes a novel approach where data is collected in a location predominantly featuring green screens. We introduce Green-screen Augmentation (GreenAug), employing a chroma key algorithm to overlay background textures onto a green screen. Through extensive real-world empirical studies with over 850 training demonstrations and 8.2k evaluation episodes, we demonstrate that GreenAug surpasses no augmentation, standard computer vision augmentation, and prior generative augmentation methods in performance. While no algorithmic novelties are claimed, our paper advocates for a fundamental shift in data collection practices. We propose that real-world demonstrations in future research should utilise green screens, followed by the application of GreenAug. We believe GreenAug unlocks policy generalisation to visually distinct novel locations, addressing the current scene generalisation limitations in robot learning.