Gaussian Mesh Renderer for Lightweight Differentiable Rendering
This work addresses a domain-specific problem for researchers and practitioners in 3D reconstruction and rendering, offering an incremental improvement by integrating Gaussian and mesh representations.
The paper tackles the problem of slow or heavy optimization in traditional mesh-based differentiable renderers by proposing a lightweight differentiable mesh renderer that leverages 3D Gaussian Splatting's efficient rasterization, achieving smoother gradients and better optimization with smaller batch sizes.
3D Gaussian Splatting (3DGS) has enabled high-fidelity virtualization with fast rendering and optimization for novel view synthesis. On the other hand, triangle mesh models still remain a popular choice for surface reconstruction but suffer from slow or heavy optimization in traditional mesh-based differentiable renderers. To address this problem, we propose a new lightweight differentiable mesh renderer leveraging the efficient rasterization process of 3DGS, named Gaussian Mesh Renderer (GMR), which tightly integrates the Gaussian and mesh representations. Each Gaussian primitive is analytically derived from the corresponding mesh triangle, preserving structural fidelity and enabling the gradient flow. Compared to the traditional mesh renderers, our method achieves smoother gradients, which especially contributes to better optimization using smaller batch sizes with limited memory. Our implementation is available in the public GitHub repository at https://github.com/huntorochi/Gaussian-Mesh-Renderer.