Skyfall-GS: Synthesizing Immersive 3D Urban Scenes from Satellite Imagery
This enables immersive 3D urban scene creation for applications like virtual reality or urban planning, without costly 3D annotations, though it is incremental as it builds on existing diffusion models and satellite data.
The paper tackles the problem of synthesizing large-scale, explorable 3D urban scenes by synergizing satellite imagery for geometry and diffusion models for appearance, resulting in improved cross-view consistent geometry and realistic textures compared to state-of-the-art methods.
Synthesizing large-scale, explorable, and geometrically accurate 3D urban scenes is a challenging yet valuable task in providing immersive and embodied applications. The challenges lie in the lack of large-scale and high-quality real-world 3D scans for training generalizable generative models. In this paper, we take an alternative route to create large-scale 3D scenes by synergizing the readily available satellite imagery that supplies realistic coarse geometry and the open-domain diffusion model for creating high-quality close-up appearances. We propose \textbf{Skyfall-GS}, the first city-block scale 3D scene creation framework without costly 3D annotations, also featuring real-time, immersive 3D exploration. We tailor a curriculum-driven iterative refinement strategy to progressively enhance geometric completeness and photorealistic textures. Extensive experiments demonstrate that Skyfall-GS provides improved cross-view consistent geometry and more realistic textures compared to state-of-the-art approaches. Project page: https://skyfall-gs.jayinnn.dev/