Instant NuRec: Feed-Forward 3D Gaussian Reconstruction for Driving Scene Simulation
This work addresses the need for fast, per-scene tuning-free neural reconstruction for autonomous driving simulation, enabling efficient closed-loop policy evaluation.
Instant NuRec is a feed-forward neural reconstruction model that converts a short multi-view driving log into a simulatable 3D Gaussian Splatting world in a single forward pass, reconstructing a 10-20-second scene in about 1.5 seconds and achieving a PSNR 2.01 dB above the strongest baseline on the Waymo Open Dataset.
3D simulation platforms are critical for autonomous driving because they enable end-to-end policy evaluation, thereby reducing development costs and improving safety. In recent years, neural simulation has become predominant, with methods such as NuRec playing a central role; however, these methods remain relatively slow and typically require per-scene tuning. In this work, we present Instant NuRec, a feed-forward neural reconstruction model that turns a short multi-view driving log into a fully simulatable 3D Gaussian Splatting (3DGS) world in a single forward pass. The model accepts multi-view input from a calibrated camera rig and emits a layered output consisting of static and dynamic 3DGS layers, a sky cubemap, and per-camera ISP corrections, while providing native support for non-pinhole camera models via 3DGUT. It reconstructs a 10-20-second multi-camera scene in roughly 1.5 seconds and achieves a PSNR on the Waymo Open Dataset that is 2.01 dB above the strongest evaluated baseline. Instant NuRec is deeply integrated into NuRec and is compatible with AlpaSim for closed-loop simulation.