Efficient Depth-Guided Urban View Synthesis
This addresses the need for efficient and robust urban view synthesis for applications like autonomous driving, though it is incremental by building on prior generalizable methods.
The paper tackles the problem of high computational cost and dense image requirements in neural radiance fields for street view synthesis by introducing EDUS, which uses predicted geometric priors to enable generalizable synthesis from sparse inputs, achieving state-of-the-art performance in sparse view settings on KITTI-360 and Waymo datasets.
Recent advances in implicit scene representation enable high-fidelity street view novel view synthesis. However, existing methods optimize a neural radiance field for each scene, relying heavily on dense training images and extensive computation resources. To mitigate this shortcoming, we introduce a new method called Efficient Depth-Guided Urban View Synthesis (EDUS) for fast feed-forward inference and efficient per-scene fine-tuning. Different from prior generalizable methods that infer geometry based on feature matching, EDUS leverages noisy predicted geometric priors as guidance to enable generalizable urban view synthesis from sparse input images. The geometric priors allow us to apply our generalizable model directly in the 3D space, gaining robustness across various sparsity levels. Through comprehensive experiments on the KITTI-360 and Waymo datasets, we demonstrate promising generalization abilities on novel street scenes. Moreover, our results indicate that EDUS achieves state-of-the-art performance in sparse view settings when combined with fast test-time optimization.