HybridWorldSim: A Scalable and Controllable High-fidelity Simulator for Autonomous Driving
This provides a scalable and high-fidelity simulator for autonomous driving research and development, addressing key limitations in view synthesis and geometric consistency.
The paper tackles the problem of realistic and controllable simulation for autonomous driving by introducing HybridWorldSim, a hybrid framework that integrates multi-traversal neural reconstruction for static backgrounds with generative modeling for dynamic agents, surpassing previous state-of-the-art methods.
Realistic and controllable simulation is critical for advancing end-to-end autonomous driving, yet existing approaches often struggle to support novel view synthesis under large viewpoint changes or to ensure geometric consistency. We introduce HybridWorldSim, a hybrid simulation framework that integrates multi-traversal neural reconstruction for static backgrounds with generative modeling for dynamic agents. This unified design addresses key limitations of previous methods, enabling the creation of diverse and high-fidelity driving scenarios with reliable visual and spatial consistency. To facilitate robust benchmarking, we further release a new multi-traversal dataset MIRROR that captures a wide range of routes and environmental conditions across different cities. Extensive experiments demonstrate that HybridWorldSim surpasses previous state-of-the-art methods, providing a practical and scalable solution for high-fidelity simulation and a valuable resource for research and development in autonomous driving.