SynthDrive: Scalable Real2Sim2Real Sensor Simulation Pipeline for High-Fidelity Asset Generation and Driving Data Synthesis
This work addresses the need for high-fidelity driving data synthesis to improve perception training in autonomous driving, representing an incremental advancement in sensor simulation.
The paper tackles the problem of generating diverse and rare sensor data for autonomous driving by proposing a scalable real2sim2real system that automates asset mining and data synthesis, resulting in a pipeline that overcomes limitations of existing CG-based and learning-based methods.
In the field of autonomous driving, sensor simulation is essential for generating rare and diverse scenarios that are difficult to capture in real-world environments. Current solutions fall into two categories: 1) CG-based methods, such as CARLA, which lack diversity and struggle to scale to the vast array of rare cases required for robust perception training; and 2) learning-based approaches, such as NeuSim, which are limited to specific object categories (vehicles) and require extensive multi-sensor data, hindering their applicability to generic objects. To address these limitations, we propose a scalable real2sim2real system that leverages 3D generation to automate asset mining, generation, and rare-case data synthesis.