CSGaussian: Progressive Rate-Distortion Compression and Segmentation for 3D Gaussian Splatting
This work addresses the need for efficient compression and segmentation in 3D scene representation, enabling decoder-side applications like scene editing, but it is incremental as it builds on prior rate-distortion-optimized compression methods.
The paper tackles the joint problem of compression and segmentation for 3D Gaussian Splatting, achieving significant reductions in transmission cost while maintaining high rendering quality and strong segmentation performance on datasets like LERF and 3D-OVS.
We present the first unified framework for rate-distortion-optimized compression and segmentation of 3D Gaussian Splatting (3DGS). While 3DGS has proven effective for both real-time rendering and semantic scene understanding, prior works have largely treated these tasks independently, leaving their joint consideration unexplored. Inspired by recent advances in rate-distortion-optimized 3DGS compression, this work integrates semantic learning into the compression pipeline to support decoder-side applications--such as scene editing and manipulation--that extend beyond traditional scene reconstruction and view synthesis. Our scheme features a lightweight implicit neural representation-based hyperprior, enabling efficient entropy coding of both color and semantic attributes while avoiding costly grid-based hyperprior as seen in many prior works. To facilitate compression and segmentation, we further develop compression-guided segmentation learning, consisting of quantization-aware training to enhance feature separability and a quality-aware weighting mechanism to suppress unreliable Gaussian primitives. Extensive experiments on the LERF and 3D-OVS datasets demonstrate that our approach significantly reduces transmission cost while preserving high rendering quality and strong segmentation performance.