DANI-Net: Uncalibrated Photometric Stereo by Differentiable Shadow Handling, Anisotropic Reflectance Modeling, and Neural Inverse Rendering
This work addresses the challenge of 3D reconstruction from images under unknown lighting for general materials, representing an incremental improvement over prior methods that assumed isotropic materials and used non-differentiable shadow maps.
The paper tackled the problem of uncalibrated photometric stereo for general objects with complex shapes and materials, such as anisotropic reflectance and irregular shadows, by proposing DANI-Net, an inverse rendering framework with differentiable shadow handling and anisotropic reflectance modeling, achieving superior and robust performance on multiple real-world datasets.
Uncalibrated photometric stereo (UPS) is challenging due to the inherent ambiguity brought by the unknown light. Although the ambiguity is alleviated on non-Lambertian objects, the problem is still difficult to solve for more general objects with complex shapes introducing irregular shadows and general materials with complex reflectance like anisotropic reflectance. To exploit cues from shadow and reflectance to solve UPS and improve performance on general materials, we propose DANI-Net, an inverse rendering framework with differentiable shadow handling and anisotropic reflectance modeling. Unlike most previous methods that use non-differentiable shadow maps and assume isotropic material, our network benefits from cues of shadow and anisotropic reflectance through two differentiable paths. Experiments on multiple real-world datasets demonstrate our superior and robust performance.