NeuMesh: Learning Disentangled Neural Mesh-based Implicit Field for Geometry and Texture Editing
This work addresses the need for fine-grained editing of general objects in neural rendering, which is incremental as it builds on existing implicit field methods to enhance editing functionality.
The paper tackles the problem of limited editing capabilities in neural implicit rendering by introducing NeuMesh, a mesh-based representation that disentangles geometry and texture codes, enabling mesh-guided geometry editing and precise texture operations like swapping, filling, and painting, with experiments showing superiority in representation quality and editing ability.
Very recently neural implicit rendering techniques have been rapidly evolved and shown great advantages in novel view synthesis and 3D scene reconstruction. However, existing neural rendering methods for editing purposes offer limited functionality, e.g., rigid transformation, or not applicable for fine-grained editing for general objects from daily lives. In this paper, we present a novel mesh-based representation by encoding the neural implicit field with disentangled geometry and texture codes on mesh vertices, which facilitates a set of editing functionalities, including mesh-guided geometry editing, designated texture editing with texture swapping, filling and painting operations. To this end, we develop several techniques including learnable sign indicators to magnify spatial distinguishability of mesh-based representation, distillation and fine-tuning mechanism to make a steady convergence, and the spatial-aware optimization strategy to realize precise texture editing. Extensive experiments and editing examples on both real and synthetic data demonstrate the superiority of our method on representation quality and editing ability. Code is available on the project webpage: https://zju3dv.github.io/neumesh/.