2D Triangle Splatting for Direct Differentiable Mesh Training
This work addresses rendering limitations in 3D scene reconstruction for applications like computer graphics and vision, offering an incremental improvement over existing differentiable rendering methods.
The paper tackles the challenge of improving rendering speed and advanced effects like relighting in differentiable rendering by proposing 2D Triangle Splatting, which replaces 3D Gaussian primitives with 2D triangle facelets to directly train photorealistic meshes, achieving higher fidelity than state-of-the-art Gaussian-based methods and superior visual quality in mesh reconstruction.
Differentiable rendering with 3D Gaussian primitives has emerged as a powerful method for reconstructing high-fidelity 3D scenes from multi-view images. While it offers improvements over NeRF-based methods, this representation still encounters challenges with rendering speed and advanced rendering effects, such as relighting and shadow rendering, compared to mesh-based models. In this paper, we propose 2D Triangle Splatting (2DTS), a novel method that replaces 3D Gaussian primitives with 2D triangle facelets. This representation naturally forms a discrete mesh-like structure while retaining the benefits of continuous volumetric modeling. By incorporating a compactness parameter into the triangle primitives, we enable direct training of photorealistic meshes. Our experimental results demonstrate that our triangle-based method, in its vanilla version (without compactness tuning), achieves higher fidelity compared to state-of-the-art Gaussian-based methods. Furthermore, our approach produces reconstructed meshes with superior visual quality compared to existing mesh reconstruction methods. Please visit our project page at https://gaoderender.github.io/triangle-splatting.