NeuS-PIR: Learning Relightable Neural Surface using Pre-Integrated Rendering
This work addresses the challenge of creating high-quality, relightable 3D surfaces for applications like graphics and VR, representing an incremental advancement over prior neural rendering techniques.
The paper tackles the problem of reconstructing relightable neural surfaces from multi-view images or videos by factorizing the radiance field into material and lighting components, leading to improved geometry and scene property disentanglement, with experiments showing it outperforms existing methods on synthetic and real datasets.
This paper presents a method, namely NeuS-PIR, for recovering relightable neural surfaces using pre-integrated rendering from multi-view images or video. Unlike methods based on NeRF and discrete meshes, our method utilizes implicit neural surface representation to reconstruct high-quality geometry, which facilitates the factorization of the radiance field into two components: a spatially varying material field and an all-frequency lighting representation. This factorization, jointly optimized using an adapted differentiable pre-integrated rendering framework with material encoding regularization, in turn addresses the ambiguity of geometry reconstruction and leads to better disentanglement and refinement of each scene property. Additionally, we introduced a method to distil indirect illumination fields from the learned representations, further recovering the complex illumination effect like inter-reflection. Consequently, our method enables advanced applications such as relighting, which can be seamlessly integrated with modern graphics engines. Qualitative and quantitative experiments have shown that NeuS-PIR outperforms existing methods across various tasks on both synthetic and real datasets. Source code is available at https://github.com/Sheldonmao/NeuSPIR