S-VolSDF: Sparse Multi-View Stereo Regularization of Neural Implicit Surfaces
This work addresses the challenge of 3D reconstruction from sparse images for computer vision applications, offering a hybrid approach that enhances both neural rendering and MVS performance.
The paper tackles the problem of neural implicit surface reconstruction from sparse input views by regularizing neural rendering with multi-view stereo (MVS) to address shape-radiance ambiguity, resulting in improved reconstruction quality that outperforms generic neural rendering and MVS models with only three input views.
Neural rendering of implicit surfaces performs well in 3D vision applications. However, it requires dense input views as supervision. When only sparse input images are available, output quality drops significantly due to the shape-radiance ambiguity problem. We note that this ambiguity can be constrained when a 3D point is visible in multiple views, as is the case in multi-view stereo (MVS). We thus propose to regularize neural rendering optimization with an MVS solution. The use of an MVS probability volume and a generalized cross entropy loss leads to a noise-tolerant optimization process. In addition, neural rendering provides global consistency constraints that guide the MVS depth hypothesis sampling and thus improves MVS performance. Given only three sparse input views, experiments show that our method not only outperforms generic neural rendering models by a large margin but also significantly increases the reconstruction quality of MVS models. Project page: https://hao-yu-wu.github.io/s-volsdf/.