GaMeS: Mesh-Based Adapting and Modification of Gaussian Splatting
This addresses the editing bottleneck in GS for 3D rendering applications, representing an incremental improvement.
The paper tackles the problem of conditioning Gaussian Splatting (GS) for editing by introducing GaMeS, which parameterizes Gaussian components using mesh vertices, enabling real-time rendering of editable GS with automatic adjustments during animation.
Gaussian Splatting (GS) is a novel, state-of-the-art technique for rendering points in a 3D scene by approximating their contribution to image pixels through Gaussian distributions, warranting fast training and real-time rendering. The main drawback of GS is the absence of a well-defined approach for its conditioning due to the necessity of conditioning several hundred thousand Gaussian components. To solve this, we introduce the Gaussian Mesh Splatting (GaMeS) model, which allows modification of Gaussian components in a similar way as meshes. We parameterize each Gaussian component by the vertices of the mesh face. Furthermore, our model needs mesh initialization on input or estimated mesh during training. We also define Gaussian splats solely based on their location on the mesh, allowing for automatic adjustments in position, scale, and rotation during animation. As a result, we obtain a real-time rendering of editable GS.