Quantile Rendering: Efficiently Embedding High-dimensional Feature on 3D Gaussian Splatting
This addresses the problem of real-time, high-fidelity 3D open-vocabulary segmentation for computer vision applications, representing a strong specific gain rather than a foundational advancement.
The paper tackles the challenge of efficiently rendering high-dimensional features for open-vocabulary segmentation in 3D Gaussian Splatting, achieving a ~43.7x speedup on 512-D feature maps while outperforming state-of-the-art methods.
Recent advancements in computer vision have successfully extended Open-vocabulary segmentation (OVS) to the 3D domain by leveraging 3D Gaussian Splatting (3D-GS). Despite this progress, efficiently rendering the high-dimensional features required for open-vocabulary queries poses a significant challenge. Existing methods employ codebooks or feature compression, causing information loss, thereby degrading segmentation quality. To address this limitation, we introduce Quantile Rendering (Q-Render), a novel rendering strategy for 3D Gaussians that efficiently handles high-dimensional features while maintaining high fidelity. Unlike conventional volume rendering, which densely samples all 3D Gaussians intersecting each ray, Q-Render sparsely samples only those with dominant influence along the ray. By integrating Q-Render into a generalizable 3D neural network, we also propose Gaussian Splatting Network (GS-Net), which predicts Gaussian features in a generalizable manner. Extensive experiments on ScanNet and LeRF demonstrate that our framework outperforms state-of-the-art methods, while enabling real-time rendering with an approximate ~43.7x speedup on 512-D feature maps. Code will be made publicly available.