A Neural Rendering Framework for Free-Viewpoint Relighting
This work addresses the limitation of existing neural rendering methods in handling relighting, offering a practical solution for free-viewpoint applications in computer graphics and vision.
The paper tackles the problem of simultaneous view synthesis and relighting from multi-view images by introducing a Relightable Neural Renderer (RNR) that models physical rendering processes, improving both relighting capabilities and view synthesis quality.
We present a novel Relightable Neural Renderer (RNR) for simultaneous view synthesis and relighting using multi-view image inputs. Existing neural rendering (NR) does not explicitly model the physical rendering process and hence has limited capabilities on relighting. RNR instead models image formation in terms of environment lighting, object intrinsic attributes, and light transport function (LTF), each corresponding to a learnable component. In particular, the incorporation of a physically based rendering process not only enables relighting but also improves the quality of view synthesis. Comprehensive experiments on synthetic and real data show that RNR provides a practical and effective solution for conducting free-viewpoint relighting.