GA-Drive: Geometry-Appearance Decoupled Modeling for Free-viewpoint Driving Scene Generatio
This work addresses the need for realistic and customizable driving simulators for training and evaluating autonomous driving systems, representing a novel method for a known bottleneck in simulation generation.
The paper tackles the problem of generating free-viewpoint, editable, and high-fidelity driving scenes for autonomous driving simulation by proposing GA-Drive, which decouples geometry and appearance to synthesize novel camera views along user-specified trajectories, resulting in substantial improvements over existing methods in metrics like NTA-IoU, NTL-IoU, and FID scores.
A free-viewpoint, editable, and high-fidelity driving simulator is crucial for training and evaluating end-to-end autonomous driving systems. In this paper, we present GA-Drive, a novel simulation framework capable of generating camera views along user-specified novel trajectories through Geometry-Appearance Decoupling and Diffusion-Based Generation. Given a set of images captured along a recorded trajectory and the corresponding scene geometry, GA-Drive synthesizes novel pseudo-views using geometry information. These pseudo-views are then transformed into photorealistic views using a trained video diffusion model. In this way, we decouple the geometry and appearance of scenes. An advantage of such decoupling is its support for appearance editing via state-of-the-art video-to-video editing techniques, while preserving the underlying geometry, enabling consistent edits across both original and novel trajectories. Extensive experiments demonstrate that GA-Drive substantially outperforms existing methods in terms of NTA-IoU, NTL-IoU, and FID scores.