Inverse Rendering using Multi-Bounce Path Tracing and Reservoir Sampling
This work addresses the problem of realistic scene reconstruction and editing for computer graphics and vision applications, representing an incremental improvement over previous methods by incorporating more accurate physical models.
The authors tackled the problem of inverse rendering from multi-view images by developing MIRReS, a two-stage framework that reconstructs explicit geometry, material, and lighting using multi-bounce path tracing and reservoir sampling, achieving state-of-the-art performance in decomposition results, especially in challenging scenarios with complex shadows.
We present MIRReS, a novel two-stage inverse rendering framework that jointly reconstructs and optimizes the explicit geometry, material, and lighting from multi-view images. Unlike previous methods that rely on implicit irradiance fields or simplified path tracing algorithms, our method extracts an explicit geometry (triangular mesh) in stage one, and introduces a more realistic physically-based inverse rendering model that utilizes multi-bounce path tracing and Monte Carlo integration. By leveraging multi-bounce path tracing, our method effectively estimates indirect illumination, including self-shadowing and internal reflections, which improves the intrinsic decomposition of shape, material, and lighting. Moreover, we incorporate reservoir sampling into our framework to address the noise in Monte Carlo integration, enhancing convergence and facilitating gradient-based optimization with low sample counts. Through qualitative and quantitative evaluation of several scenarios, especially in challenging scenarios with complex shadows, we demonstrate that our method achieves state-of-the-art performance on decomposition results. Additionally, our optimized explicit geometry enables applications such as scene editing, relighting, and material editing with modern graphics engines or CAD software. The source code is available at https://brabbitdousha.github.io/MIRReS/