ProbNVS: Fast Novel View Synthesis with Learned Probability-Guided Sampling
This addresses the problem of slow rendering speeds in novel view synthesis for applications like virtual reality and 3D reconstruction, though it is incremental as it builds on existing neural volumetric rendering methods.
The paper tackles the inefficiency of novel view synthesis methods by proposing a framework that uses learned depth probability distributions to guide sampling, reducing sampling points while maintaining quality. It achieves 15 to 40 times faster rendering compared to state-of-the-art baselines with comparable high-quality results.
Existing state-of-the-art novel view synthesis methods rely on either fairly accurate 3D geometry estimation or sampling of the entire space for neural volumetric rendering, which limit the overall efficiency. In order to improve the rendering efficiency by reducing sampling points without sacrificing rendering quality, we propose to build a novel view synthesis framework based on learned MVS priors that enables general, fast and photo-realistic view synthesis simultaneously. Specifically, fewer but important points are sampled under the guidance of depth probability distributions extracted from the learned MVS architecture. Based on the learned probability-guided sampling, a neural volume rendering module is elaborately devised to fully aggregate source view information as well as the learned scene structures to synthesize photorealistic target view images. Finally, the rendering results in uncertain, occluded and unreferenced regions can be further improved by incorporating a confidence-aware refinement module. Experiments show that our method achieves 15 to 40 times faster rendering compared to state-of-the-art baselines, with strong generalization capacity and comparable high-quality novel view synthesis performance.