ComSim: Building Scalable Real-World Robot Data Generation via Compositional Simulation
This work addresses the data scarcity problem for robotics researchers and practitioners by providing a scalable method to generate diverse training data that bridges the simulation-to-reality gap.
The paper tackles the challenge of acquiring large-scale, high-quality training data for robotics by proposing Compositional Simulation, a hybrid approach combining classical and neural simulation to generate accurate action-video pairs with real-world consistency. The method significantly reduces the sim2real domain gap, leading to higher success rates in real-world policy model training.
Recent advancements in foundational models, such as large language models and world models, have greatly enhanced the capabilities of robotics, enabling robots to autonomously perform complex tasks. However, acquiring large-scale, high-quality training data for robotics remains a challenge, as it often requires substantial manual effort and is limited in its coverage of diverse real-world environments. To address this, we propose a novel hybrid approach called Compositional Simulation, which combines classical simulation and neural simulation to generate accurate action-video pairs while maintaining real-world consistency. Our approach utilizes a closed-loop real-sim-real data augmentation pipeline, leveraging a small amount of real-world data to generate diverse, large-scale training datasets that cover a broader spectrum of real-world scenarios. We train a neural simulator to transform classical simulation videos into real-world representations, improving the accuracy of policy models trained in real-world environments. Through extensive experiments, we demonstrate that our method significantly reduces the sim2real domain gap, resulting in higher success rates in real-world policy model training. Our approach offers a scalable solution for generating robust training data and bridging the gap between simulated and real-world robotics.