SERES: Semantic-aware neural reconstruction from sparse views
This work addresses the challenge of severe radiance ambiguity in sparse-view 3D reconstruction for computer vision applications, representing an incremental improvement over existing methods.
The paper tackles the problem of generating high-fidelity 3D models from sparse images by proposing a semantic-aware neural reconstruction method, resulting in reductions in average chamfer distances by 44% for SparseNeuS and 20% for VolRecon on the DTU dataset.
We propose a semantic-aware neural reconstruction method to generate 3D high-fidelity models from sparse images. To tackle the challenge of severe radiance ambiguity caused by mismatched features in sparse input, we enrich neural implicit representations by adding patch-based semantic logits that are optimized together with the signed distance field and the radiance field. A novel regularization based on the geometric primitive masks is introduced to mitigate shape ambiguity. The performance of our approach has been verified in experimental evaluation. The average chamfer distances of our reconstruction on the DTU dataset can be reduced by 44% for SparseNeuS and 20% for VolRecon. When working as a plugin for those dense reconstruction baselines such as NeuS and Neuralangelo, the average error on the DTU dataset can be reduced by 69% and 68% respectively.