PS-NeRF: Neural Inverse Rendering for Multi-view Photometric Stereo
This work addresses the challenge of accumulated errors in traditional MVPS for computer vision and graphics applications, offering an incremental improvement through a neural approach.
The paper tackles the problem of multi-view photometric stereo (MVPS) by proposing a neural inverse rendering method that jointly estimates geometry, materials, and lights from multi-view images of non-Lambertian objects under unknown directional lighting, achieving far more accurate shape reconstruction than existing methods.
Traditional multi-view photometric stereo (MVPS) methods are often composed of multiple disjoint stages, resulting in noticeable accumulated errors. In this paper, we present a neural inverse rendering method for MVPS based on implicit representation. Given multi-view images of a non-Lambertian object illuminated by multiple unknown directional lights, our method jointly estimates the geometry, materials, and lights. Our method first employs multi-light images to estimate per-view surface normal maps, which are used to regularize the normals derived from the neural radiance field. It then jointly optimizes the surface normals, spatially-varying BRDFs, and lights based on a shadow-aware differentiable rendering layer. After optimization, the reconstructed object can be used for novel-view rendering, relighting, and material editing. Experiments on both synthetic and real datasets demonstrate that our method achieves far more accurate shape reconstruction than existing MVPS and neural rendering methods. Our code and model can be found at https://ywq.github.io/psnerf.