A Convolutional Neural Deferred Shader for Physics Based Rendering
This work addresses computational and data challenges in neural rendering for photorealistic shading and relighting, offering an incremental improvement over existing methods.
The paper tackles the problem of high computational cost and data inefficiency in neural rendering for photorealistic shading and relighting by introducing pbnds+, a physics-based neural deferred shading pipeline that uses convolutional neural networks to reduce parameters and improve performance, with energy regularization to handle dark scenes; it outperforms classical baselines, a state-of-the-art neural shading model, and a diffusion-based method in experiments.
Recent advances in neural rendering have achieved impressive results on photorealistic shading and relighting, by using a multilayer perceptron (MLP) as a regression model to learn the rendering equation from a real-world dataset. Such methods show promise for photorealistically relighting real-world objects, which is difficult to classical rendering, as there is no easy-obtained material ground truth. However, significant challenges still remain the dense connections in MLPs result in a large number of parameters, which requires high computation resources, complicating the training, and reducing performance during rendering. Data driven approaches require large amounts of training data for generalization; unbalanced data might bias the model to ignore the unusual illumination conditions, e.g. dark scenes. This paper introduces pbnds+: a novel physics-based neural deferred shading pipeline utilizing convolution neural networks to decrease the parameters and improve the performance in shading and relighting tasks; Energy regularization is also proposed to restrict the model reflection during dark illumination. Extensive experiments demonstrate that our approach outperforms classical baselines, a state-of-the-art neural shading model, and a diffusion-based method.