TraM-NeRF: Tracing Mirror and Near-Perfect Specular Reflections through Neural Radiance Fields
This addresses a specific challenge in 3D scene reconstruction for computer vision and graphics applications, offering an incremental improvement over existing NeRF methods.
The paper tackles the problem of rendering mirror-like specular reflections in Neural Radiance Fields (NeRF), which cause artifacts, by introducing a reflection tracing method that models reflections physically and uses Monte-Carlo estimation for efficient volume rendering. The result is superior performance compared to previous state-of-the-art approaches, enabling consistent scene representations.
Implicit representations like Neural Radiance Fields (NeRF) showed impressive results for photorealistic rendering of complex scenes with fine details. However, ideal or near-perfectly specular reflecting objects such as mirrors, which are often encountered in various indoor scenes, impose ambiguities and inconsistencies in the representation of the reconstructed scene leading to severe artifacts in the synthesized renderings. In this paper, we present a novel reflection tracing method tailored for the involved volume rendering within NeRF that takes these mirror-like objects into account while avoiding the cost of straightforward but expensive extensions through standard path tracing. By explicitly modeling the reflection behavior using physically plausible materials and estimating the reflected radiance with Monte-Carlo methods within the volume rendering formulation, we derive efficient strategies for importance sampling and the transmittance computation along rays from only few samples. We show that our novel method enables the training of consistent representations of such challenging scenes and achieves superior results in comparison to previous state-of-the-art approaches.