ReinDriveGen: Reinforcement Post-Training for Out-of-Distribution Driving Scene Generation
This addresses the need for simulating rare driving events to enhance autonomous vehicle testing, though it is incremental as it builds on existing video diffusion models.
The paper tackles the problem of generating realistic driving videos for safety-critical out-of-distribution scenarios by proposing ReinDriveGen, a framework that uses reinforcement learning post-training to improve video synthesis quality, achieving state-of-the-art results on novel ego viewpoint synthesis.
We present ReinDriveGen, a framework that enables full controllability over dynamic driving scenes, allowing users to freely edit actor trajectories to simulate safety-critical corner cases such as front-vehicle collisions, drifting cars, vehicles spinning out of control, pedestrians jaywalking, and cyclists cutting across lanes. Our approach constructs a dynamic 3D point cloud scene from multi-frame LiDAR data, introduces a vehicle completion module to reconstruct full 360° geometry from partial observations, and renders the edited scene into 2D condition images that guide a video diffusion model to synthesize realistic driving videos. Since such edited scenarios inevitably fall outside the training distribution, we further propose an RL-based post-training strategy with a pairwise preference model and a pairwise reward mechanism, enabling robust quality improvement under out-of-distribution conditions without ground-truth supervision. Extensive experiments demonstrate that ReinDriveGen outperforms existing approaches on edited driving scenarios and achieves state-of-the-art results on novel ego viewpoint synthesis.