GSFF-SLAM: 3D Semantic Gaussian Splatting SLAM via Feature Field
This work addresses the challenge of robust semantic mapping in real-world environments for autonomous robots, representing an incremental improvement over existing methods.
The paper tackles the problem of semantic-aware 3D scene reconstruction for autonomous robots by proposing GSFF-SLAM, a system that uses 3D Gaussian Splatting and feature fields to handle sparse and noisy 2D priors, achieving state-of-the-art semantic segmentation with 95.03% mIoU and up to 2.9x speedup.
Semantic-aware 3D scene reconstruction is essential for autonomous robots to perform complex interactions. Semantic SLAM, an online approach, integrates pose tracking, geometric reconstruction, and semantic mapping into a unified framework, shows significant potential. However, existing systems, which rely on 2D ground truth priors for supervision, are often limited by the sparsity and noise of these signals in real-world environments. To address this challenge, we propose GSFF-SLAM, a novel dense semantic SLAM system based on 3D Gaussian Splatting that leverages feature fields to achieve joint rendering of appearance, geometry, and N-dimensional semantic features. By independently optimizing feature gradients, our method supports semantic reconstruction using various forms of 2D priors, particularly sparse and noisy signals. Experimental results demonstrate that our approach outperforms previous methods in both tracking accuracy and photorealistic rendering quality. When utilizing 2D ground truth priors, GSFF-SLAM achieves state-of-the-art semantic segmentation performance with 95.03\% mIoU, while achieving up to 2.9$\times$ speedup with only marginal performance degradation.