Learning Object-Centric Neural Scattering Functions for Free-Viewpoint Relighting and Scene Composition
This addresses the challenge of relighting and modeling translucent objects in neural implicit methods for vision and graphics, representing an incremental improvement over prior neural inverse rendering approaches.
The paper tackles the problem of photorealistic object appearance modeling from 2D images by proposing Object-Centric Neural Scattering Functions (OSFs), which enable free-viewpoint relighting and scene composition for both opaque and translucent objects, with experiments on real and synthetic data showing accurate reconstructions.
Photorealistic object appearance modeling from 2D images is a constant topic in vision and graphics. While neural implicit methods (such as Neural Radiance Fields) have shown high-fidelity view synthesis results, they cannot relight the captured objects. More recent neural inverse rendering approaches have enabled object relighting, but they represent surface properties as simple BRDFs, and therefore cannot handle translucent objects. We propose Object-Centric Neural Scattering Functions (OSFs) for learning to reconstruct object appearance from only images. OSFs not only support free-viewpoint object relighting, but also can model both opaque and translucent objects. While accurately modeling subsurface light transport for translucent objects can be highly complex and even intractable for neural methods, OSFs learn to approximate the radiance transfer from a distant light to an outgoing direction at any spatial location. This approximation avoids explicitly modeling complex subsurface scattering, making learning a neural implicit model tractable. Experiments on real and synthetic data show that OSFs accurately reconstruct appearances for both opaque and translucent objects, allowing faithful free-viewpoint relighting as well as scene composition.