REdiSplats: Ray Tracing for Editable Gaussian Splatting
This addresses the problem of making GS more editable and physically realistic for 3D rendering applications, though it appears incremental as it builds on prior work incorporating ray tracing or meshes into GS.
The paper tackles the limitations of Gaussian Splatting (GS) in handling varying light conditions, manual adjustments, and physical simulation by introducing REdiSplats, which uses ray tracing and a mesh-based representation of flat Gaussians, enabling integration with 3D tools like Blender and Nvdiffrast.
Gaussian Splatting (GS) has become one of the most important neural rendering algorithms. GS represents 3D scenes using Gaussian components with trainable color and opacity. This representation achieves high-quality renderings with fast inference. Regrettably, it is challenging to integrate such a solution with varying light conditions, including shadows and light reflections, manual adjustments, and a physical engine. Recently, a few approaches have appeared that incorporate ray-tracing or mesh primitives into GS to address some of these caveats. However, no such solution can simultaneously solve all the existing limitations of the classical GS. Consequently, we introduce REdiSplats, which employs ray tracing and a mesh-based representation of flat 3D Gaussians. In practice, we model the scene using flat Gaussian distributions parameterized by the mesh. We can leverage fast ray tracing and control Gaussian modification by adjusting the mesh vertices. Moreover, REdiSplats allows modeling of light conditions, manual adjustments, and physical simulation. Furthermore, we can render our models using 3D tools such as Blender or Nvdiffrast, which opens the possibility of integrating them with all existing 3D graphics techniques dedicated to mesh representations.