Fast and Physically-based Neural Explicit Surface for Relightable Human Avatars
This work addresses the need for fast and physically-based avatars in AR/VR applications, representing an incremental improvement over existing neural implicit methods.
The paper tackles the problem of efficiently modeling relightable human avatars from sparse-view videos by introducing PhyNES, which uses neural explicit surfaces to achieve relighting quality comparable to state-of-the-art methods while significantly improving rendering speed, memory efficiency, and reconstruction quality.
Efficiently modeling relightable human avatars from sparse-view videos is crucial for AR/VR applications. Current methods use neural implicit representations to capture dynamic geometry and reflectance, which incur high costs due to the need for dense sampling in volume rendering. To overcome these challenges, we introduce Physically-based Neural Explicit Surface (PhyNES), which employs compact neural material maps based on the Neural Explicit Surface (NES) representation. PhyNES organizes human models in a compact 2D space, enhancing material disentanglement efficiency. By connecting Signed Distance Fields to explicit surfaces, PhyNES enables efficient geometry inference around a parameterized human shape model. This approach models dynamic geometry, texture, and material maps as 2D neural representations, enabling efficient rasterization. PhyNES effectively captures physical surface attributes under varying illumination, enabling real-time physically-based rendering. Experiments show that PhyNES achieves relighting quality comparable to SOTA methods while significantly improving rendering speed, memory efficiency, and reconstruction quality.