TSGaussian: Semantic and Depth-Guided Target-Specific Gaussian Splatting from Sparse Views
This addresses the need for target-specific 3D reconstruction from limited viewpoints, which is an incremental improvement over existing Gaussian Splatting methods.
The paper tackles the problem of reconstructing specified targets with complex structures from sparse views in 3D Gaussian Splatting, introducing TSGaussian which combines semantic constraints and depth priors to achieve superior novel view synthesis results on standard and new challenging datasets.
Recent advances in Gaussian Splatting have significantly advanced the field, achieving both panoptic and interactive segmentation of 3D scenes. However, existing methodologies often overlook the critical need for reconstructing specified targets with complex structures from sparse views. To address this issue, we introduce TSGaussian, a novel framework that combines semantic constraints with depth priors to avoid geometry degradation in challenging novel view synthesis tasks. Our approach prioritizes computational resources on designated targets while minimizing background allocation. Bounding boxes from YOLOv9 serve as prompts for Segment Anything Model to generate 2D mask predictions, ensuring semantic accuracy and cost efficiency. TSGaussian effectively clusters 3D gaussians by introducing a compact identity encoding for each Gaussian ellipsoid and incorporating 3D spatial consistency regularization. Leveraging these modules, we propose a pruning strategy to effectively reduce redundancy in 3D gaussians. Extensive experiments demonstrate that TSGaussian outperforms state-of-the-art methods on three standard datasets and a new challenging dataset we collected, achieving superior results in novel view synthesis of specific objects. Code is available at: https://github.com/leon2000-ai/TSGaussian.