CF3: Compact and Fast 3D Feature Fields
This work addresses computational bottlenecks in 3D scene representation for applications like computer vision and graphics, though it is incremental as it builds on existing 3D Gaussian Splatting methods.
The paper tackles the computational inefficiency of incorporating 2D foundation model features into 3D Gaussian Splatting by proposing CF3, a top-down pipeline that uses fast weighted fusion and adaptive sparsification, achieving a competitive 3D feature field with only 5% of the Gaussians compared to Feature-3DGS.
3D Gaussian Splatting (3DGS) has begun incorporating rich information from 2D foundation models. However, most approaches rely on a bottom-up optimization process that treats raw 2D features as ground truth, incurring increased computational costs. We propose a top-down pipeline for constructing compact and fast 3D Gaussian feature fields, namely, CF3. We first perform a fast weighted fusion of multi-view 2D features with pre-trained Gaussians. This approach enables training a per-Gaussian autoencoder directly on the lifted features, instead of training autoencoders in the 2D domain. As a result, the autoencoder better aligns with the feature distribution. More importantly, we introduce an adaptive sparsification method that optimizes the Gaussian attributes of the feature field while pruning and merging the redundant Gaussians, constructing an efficient representation with preserved geometric details. Our approach achieves a competitive 3D feature field using as little as 5% of the Gaussians compared to Feature-3DGS.