Sketch-guided Cage-based 3D Gaussian Splatting Deformation
This work addresses the problem of intuitive and fine-grained 3D model editing for users in computer graphics and vision, representing an incremental advancement over existing editing methods.
The paper tackles the challenge of fine-grained deformation control in 3D Gaussian Splatting models by introducing a sketch-guided system that allows intuitive geometry modifications via silhouette sketches, achieving precise and semantically plausible deformations as demonstrated in experiments.
3D Gaussian Splatting (GS) is one of the most promising novel 3D representations that has received great interest in computer graphics and computer vision. While various systems have introduced editing capabilities for 3D GS, such as those guided by text prompts, fine-grained control over deformation remains an open challenge. In this work, we present a novel sketch-guided 3D GS deformation system that allows users to intuitively modify the geometry of a 3D GS model by drawing a silhouette sketch from a single viewpoint. Our approach introduces a new deformation method that combines cage-based deformations with a variant of Neural Jacobian Fields, enabling precise, fine-grained control. Additionally, it leverages large-scale 2D diffusion priors and ControlNet to ensure the generated deformations are semantically plausible. Through a series of experiments, we demonstrate the effectiveness of our method and showcase its ability to animate static 3D GS models as one of its key applications.