Training-Free Hierarchical Scene Understanding for Gaussian Splatting with Superpoint Graphs
This addresses inefficiencies and view inconsistencies in 3D scene understanding for applications like robotics and AR/VR, though it is incremental as it builds on existing 3DGS methods.
The paper tackles the problem of inconsistent 3D semantics in open-vocabulary scene understanding for 3D Gaussian Splatting by introducing a training-free framework that constructs a superpoint graph from Gaussian primitives, achieving state-of-the-art segmentation performance with semantic field reconstruction over 30× faster.
Bridging natural language and 3D geometry is a crucial step toward flexible, language-driven scene understanding. While recent advances in 3D Gaussian Splatting (3DGS) have enabled fast and high-quality scene reconstruction, research has also explored incorporating open-vocabulary understanding into 3DGS. However, most existing methods require iterative optimization over per-view 2D semantic feature maps, which not only results in inefficiencies but also leads to inconsistent 3D semantics across views. To address these limitations, we introduce a training-free framework that constructs a superpoint graph directly from Gaussian primitives. The superpoint graph partitions the scene into spatially compact and semantically coherent regions, forming view-consistent 3D entities and providing a structured foundation for open-vocabulary understanding. Based on the graph structure, we design an efficient reprojection strategy that lifts 2D semantic features onto the superpoints, avoiding costly multi-view iterative training. The resulting representation ensures strong 3D semantic coherence and naturally supports hierarchical understanding, enabling both coarse- and fine-grained open-vocabulary perception within a unified semantic field. Extensive experiments demonstrate that our method achieves state-of-the-art open-vocabulary segmentation performance, with semantic field reconstruction completed over $30\times$ faster. Our code will be available at https://github.com/Atrovast/THGS.