ENVIDR: Implicit Differentiable Renderer with Neural Environment Lighting
This work addresses a specific problem in neural rendering for glossy surfaces, offering incremental improvements for applications like material editing and scene relighting.
The authors tackled the challenge of accurately representing glossy surfaces in neural rendering by introducing ENVIDR, a framework that uses a novel neural renderer and SDF-based model to handle specular reflections, outperforming state-of-the-art methods on shiny scenes.
Recent advances in neural rendering have shown great potential for reconstructing scenes from multiview images. However, accurately representing objects with glossy surfaces remains a challenge for existing methods. In this work, we introduce ENVIDR, a rendering and modeling framework for high-quality rendering and reconstruction of surfaces with challenging specular reflections. To achieve this, we first propose a novel neural renderer with decomposed rendering components to learn the interaction between surface and environment lighting. This renderer is trained using existing physically based renderers and is decoupled from actual scene representations. We then propose an SDF-based neural surface model that leverages this learned neural renderer to represent general scenes. Our model additionally synthesizes indirect illuminations caused by inter-reflections from shiny surfaces by marching surface-reflected rays. We demonstrate that our method outperforms state-of-art methods on challenging shiny scenes, providing high-quality rendering of specular reflections while also enabling material editing and scene relighting.