3DGS-VBench: A Comprehensive Video Quality Evaluation Benchmark for 3DGS Compression
This work addresses the problem of evaluating distortions in compressed 3DGS models for researchers and practitioners in computer vision and graphics, providing a foundational benchmark for future compression and quality assessment research.
The paper tackles the lack of systematic quality assessment for compressed 3D Gaussian Splatting (3DGS) models by introducing 3DGS-VBench, a large-scale video quality evaluation benchmark with 660 compressed models and sequences from 11 scenes, annotated by 50 participants to provide MOS scores.
3D Gaussian Splatting (3DGS) enables real-time novel view synthesis with high visual fidelity, but its substantial storage requirements hinder practical deployment, prompting state-of-the-art (SOTA) 3DGS methods to incorporate compression modules. However, these 3DGS generative compression techniques introduce unique distortions lacking systematic quality assessment research. To this end, we establish 3DGS-VBench, a large-scale Video Quality Assessment (VQA) Dataset and Benchmark with 660 compressed 3DGS models and video sequences generated from 11 scenes across 6 SOTA 3DGS compression algorithms with systematically designed parameter levels. With annotations from 50 participants, we obtained MOS scores with outlier removal and validated dataset reliability. We benchmark 6 3DGS compression algorithms on storage efficiency and visual quality, and evaluate 15 quality assessment metrics across multiple paradigms. Our work enables specialized VQA model training for 3DGS, serving as a catalyst for compression and quality assessment research. The dataset is available at https://github.com/YukeXing/3DGS-VBench.