RT-GS: Gaussian Splatting with Reflection and Transmittance Primitives
This addresses the limitation of existing methods in handling specular light interactions for 3D scene reconstruction, though it is incremental by extending Gaussian Splatting with physical modeling.
The paper tackles the problem of modeling specular reflections and transmittance in Gaussian Splatting for realistic novel view synthesis, achieving significant qualitative improvements in complex environments.
Gaussian Splatting is a powerful tool for reconstructing diffuse scenes, but it struggles to simultaneously model specular reflections and the appearance of objects behind semi-transparent surfaces. These specular reflections and transmittance are essential for realistic novel view synthesis, and existing methods do not properly incorporate the underlying physical processes to simulate them. To address this issue, we propose RT-GS, a unified framework that integrates a microfacet material model and ray tracing to jointly model specular reflection and transmittance in Gaussian Splatting. We accomplish this by using separate Gaussian primitives for reflections and transmittance, which allow modeling distant reflections and reconstructing objects behind transparent surfaces concurrently. We utilize a differentiable ray tracing framework to obtain the specular reflection and transmittance appearance. Our experiments demonstrate that our method successfully produces reflections and recovers objects behind transparent surfaces in complex environments, achieving significant qualitative improvements over prior methods where these specular light interactions are prominent.