Perspective-aware fusion of incomplete depth maps and surface normals for accurate 3D reconstruction
This work addresses 3D reconstruction for applications like scanning and robotics, but it is incremental as it extends existing orthographic methods to perspective projection.
The paper tackles the problem of reconstructing 3D surfaces from incomplete depth and surface normal maps using a single perspective camera, proposing a perspective-aware log-depth fusion method that improves metric accuracy and handles missing data by inpainting with surface normals, achieving effective results on the DiLiGenT-MV dataset.
We address the problem of reconstructing 3D surfaces from depth and surface normal maps acquired by a sensor system based on a single perspective camera. Depth and normal maps can be obtained through techniques such as structured-light scanning and photometric stereo, respectively. We propose a perspective-aware log-depth fusion approach that extends existing orthographic gradient-based depth-normals fusion methods by explicitly accounting for perspective projection, leading to metrically accurate 3D reconstructions. Additionally, the method handles missing depth measurements by leveraging available surface normal information to inpaint gaps. Experiments on the DiLiGenT-MV data set demonstrate the effectiveness of our approach and highlight the importance of perspective-aware depth-normals fusion.