Robot Learning from Any Images
This work democratizes robotic data generation for researchers and practitioners by eliminating the need for additional hardware or digital assets, though it builds incrementally on existing scene recovery and blending techniques.
The authors tackled the problem of generating interactive robotic environments from any single image, enabling massive visuomotor demonstration production from diverse sources like camera captures and Internet images, with results showing data generation within minutes.
We introduce RoLA, a framework that transforms any in-the-wild image into an interactive, physics-enabled robotic environment. Unlike previous methods, RoLA operates directly on a single image without requiring additional hardware or digital assets. Our framework democratizes robotic data generation by producing massive visuomotor robotic demonstrations within minutes from a wide range of image sources, including camera captures, robotic datasets, and Internet images. At its core, our approach combines a novel method for single-view physical scene recovery with an efficient visual blending strategy for photorealistic data collection. We demonstrate RoLA's versatility across applications like scalable robotic data generation and augmentation, robot learning from Internet images, and single-image real-to-sim-to-real systems for manipulators and humanoids. Video results are available at https://sihengz02.github.io/RoLA .