Neural Inverse Rendering from Propagating Light
This enables more accurate 3D reconstruction in challenging lighting conditions for applications like computer vision and graphics.
The authors tackled the problem of 3D reconstruction from multi-view videos of propagating light in the presence of strong indirect illumination, achieving state-of-the-art results with a system that accurately accounts for direct and indirect light transport effects.
We present the first system for physically based, neural inverse rendering from multi-viewpoint videos of propagating light. Our approach relies on a time-resolved extension of neural radiance caching -- a technique that accelerates inverse rendering by storing infinite-bounce radiance arriving at any point from any direction. The resulting model accurately accounts for direct and indirect light transport effects and, when applied to captured measurements from a flash lidar system, enables state-of-the-art 3D reconstruction in the presence of strong indirect light. Further, we demonstrate view synthesis of propagating light, automatic decomposition of captured measurements into direct and indirect components, as well as novel capabilities such as multi-view time-resolved relighting of captured scenes.