High-Quality RGB-D Reconstruction via Multi-View Uncalibrated Photometric Stereo and Gradient-SDF
This work addresses the need for fine-detailed reconstructions in applications like robotics or AR/VR, but it is incremental as it builds on existing RGB-D and photometric stereo techniques with a novel optimization approach.
The paper tackles the problem of high-quality 3D reconstruction from RGB-D data without relying on pre-calculated camera poses, by jointly optimizing camera pose, lighting, albedo, and surface normals using a gradient signed distance field, and demonstrates improved geometry recovery and pose accuracy over state-of-the-art methods on synthetic and real-world datasets.
Fine-detailed reconstructions are in high demand in many applications. However, most of the existing RGB-D reconstruction methods rely on pre-calculated accurate camera poses to recover the detailed surface geometry, where the representation of a surface needs to be adapted when optimizing different quantities. In this paper, we present a novel multi-view RGB-D based reconstruction method that tackles camera pose, lighting, albedo, and surface normal estimation via the utilization of a gradient signed distance field (gradient-SDF). The proposed method formulates the image rendering process using specific physically-based model(s) and optimizes the surface's quantities on the actual surface using its volumetric representation, as opposed to other works which estimate surface quantities only near the actual surface. To validate our method, we investigate two physically-based image formation models for natural light and point light source applications. The experimental results on synthetic and real-world datasets demonstrate that the proposed method can recover high-quality geometry of the surface more faithfully than the state-of-the-art and further improves the accuracy of estimated camera poses.