AutoScape: Geometry-Consistent Long-Horizon Scene Generation
This addresses the challenge of generating realistic and geometrically consistent driving videos for applications like autonomous vehicle simulation, representing a strong specific gain rather than a foundational advancement.
The paper tackles the problem of generating long-horizon driving scenes by proposing AutoScape, a framework that uses a novel RGB-D diffusion model to create geometrically consistent keyframes and interpolates them into dense videos, resulting in improvements of 48.6% in FID and 43.0% in FVD scores over prior state-of-the-art.
This paper proposes AutoScape, a long-horizon driving scene generation framework. At its core is a novel RGB-D diffusion model that iteratively generates sparse, geometrically consistent keyframes, serving as reliable anchors for the scene's appearance and geometry. To maintain long-range geometric consistency, the model 1) jointly handles image and depth in a shared latent space, 2) explicitly conditions on the existing scene geometry (i.e., rendered point clouds) from previously generated keyframes, and 3) steers the sampling process with a warp-consistent guidance. Given high-quality RGB-D keyframes, a video diffusion model then interpolates between them to produce dense and coherent video frames. AutoScape generates realistic and geometrically consistent driving videos of over 20 seconds, improving the long-horizon FID and FVD scores over the prior state-of-the-art by 48.6\% and 43.0\%, respectively.