RaNeuS: Ray-adaptive Neural Surface Reconstruction
This addresses the challenge of detailed surface reconstruction in 3D computer vision, with applications in VR/AR and robotics, though it appears incremental as it builds on existing SDF-based NeRF methods.
The paper tackles the problem of reconstructing detailed 3D surfaces from neural radiance fields (NeRF), where existing methods fail to capture small-scale details like leaves and textiles. It introduces RaNeuS, which adaptively adjusts regularization and projection during optimization, achieving state-of-the-art results on both novel view synthesis and geometric reconstruction.
Our objective is to leverage a differentiable radiance field \eg NeRF to reconstruct detailed 3D surfaces in addition to producing the standard novel view renderings. There have been related methods that perform such tasks, usually by utilizing a signed distance field (SDF). However, the state-of-the-art approaches still fail to correctly reconstruct the small-scale details, such as the leaves, ropes, and textile surfaces. Considering that different methods formulate and optimize the projection from SDF to radiance field with a globally constant Eikonal regularization, we improve with a ray-wise weighting factor to prioritize the rendering and zero-crossing surface fitting on top of establishing a perfect SDF. We propose to adaptively adjust the regularization on the signed distance field so that unsatisfying rendering rays won't enforce strong Eikonal regularization which is ineffective, and allow the gradients from regions with well-learned radiance to effectively back-propagated to the SDF. Consequently, balancing the two objectives in order to generate accurate and detailed surfaces. Additionally, concerning whether there is a geometric bias between the zero-crossing surface in SDF and rendering points in the radiance field, the projection becomes adjustable as well depending on different 3D locations during optimization. Our proposed \textit{RaNeuS} are extensively evaluated on both synthetic and real datasets, achieving state-of-the-art results on both novel view synthesis and geometric reconstruction.