Texture-GS: Disentangling the Geometry and Texture for 3D Gaussian Splatting Editing
This addresses the need for flexible editing in 3D reconstruction for applications like graphics and VR, though it is incremental as it builds on existing 3D Gaussian splatting methods.
The paper tackles the problem of editing 3D Gaussian splatting scenes, which couples appearance and geometry, by proposing Texture-GS to disentangle them using a 2D texture mapped onto the 3D surface, enabling high-fidelity appearance editing and real-time rendering on consumer-level devices like an RTX 2080 Ti GPU.
3D Gaussian splatting, emerging as a groundbreaking approach, has drawn increasing attention for its capabilities of high-fidelity reconstruction and real-time rendering. However, it couples the appearance and geometry of the scene within the Gaussian attributes, which hinders the flexibility of editing operations, such as texture swapping. To address this issue, we propose a novel approach, namely Texture-GS, to disentangle the appearance from the geometry by representing it as a 2D texture mapped onto the 3D surface, thereby facilitating appearance editing. Technically, the disentanglement is achieved by our proposed texture mapping module, which consists of a UV mapping MLP to learn the UV coordinates for the 3D Gaussian centers, a local Taylor expansion of the MLP to efficiently approximate the UV coordinates for the ray-Gaussian intersections, and a learnable texture to capture the fine-grained appearance. Extensive experiments on the DTU dataset demonstrate that our method not only facilitates high-fidelity appearance editing but also achieves real-time rendering on consumer-level devices, e.g. a single RTX 2080 Ti GPU.