Beyond Reconstruction: A Physics Based Neural Deferred Shader for Photo-realistic Rendering
This addresses a limitation in visual effects and video game rendering by allowing parameter control, though it is incremental as it builds on existing neural rendering methods.
The paper tackles the problem of decomposing illumination and material parameters in deep learning-based rendering, enabling control over these parameters for photo-realistic shading and relighting tasks, achieving improved performance compared to classical and state-of-the-art neural models.
Deep learning based rendering has achieved major improvements in photo-realistic image synthesis, with potential applications including visual effects in movies and photo-realistic scene building in video games. However, a significant limitation is the difficulty of decomposing the illumination and material parameters, which limits such methods to reconstructing an input scene, without any possibility to control these parameters. This paper introduces a novel physics based neural deferred shading pipeline to decompose the data-driven rendering process, learn a generalizable shading function to produce photo-realistic results for shading and relighting tasks; we also propose a shadow estimator to efficiently mimic shadowing effects. Our model achieves improved performance compared to classical models and a state-of-art neural shading model, and enables generalizable photo-realistic shading from arbitrary illumination input.