AG$^2$aussian: Anchor-Graph Structured Gaussian Splatting for Instance-Level 3D Scene Understanding and Editing
This work addresses the need for semantic-aware 3D Gaussian representations to enable scene understanding and editing tasks in 3D Gaussian Splatting applications, representing an incremental improvement over existing methods.
The paper tackled the problem of noisy segmentation and messy Gaussian selection in 3D Gaussian Splatting for semantic-aware 3D scene understanding and editing by introducing AG$^2$aussian, which uses an anchor-graph structure to organize semantic features and regulate Gaussian primitives, resulting in clean and accurate instance-level Gaussian selection validated across four applications.
3D Gaussian Splatting (3DGS) has witnessed exponential adoption across diverse applications, driving a critical need for semantic-aware 3D Gaussian representations to enable scene understanding and editing tasks. Existing approaches typically attach semantic features to a collection of free Gaussians and distill the features via differentiable rendering, leading to noisy segmentation and a messy selection of Gaussians. In this paper, we introduce AG$^2$aussian, a novel framework that leverages an anchor-graph structure to organize semantic features and regulate Gaussian primitives. Our anchor-graph structure not only promotes compact and instance-aware Gaussian distributions, but also facilitates graph-based propagation, achieving a clean and accurate instance-level Gaussian selection. Extensive validation across four applications, i.e. interactive click-based query, open-vocabulary text-driven query, object removal editing, and physics simulation, demonstrates the advantages of our approach and its benefits to various applications. The experiments and ablation studies further evaluate the effectiveness of the key designs of our approach.