GSStream: 3D Gaussian Splatting based Volumetric Scene Streaming System
This work addresses the problem of real-time distribution of immersive 3D scenes for users, but it is incremental as it builds on existing 3DGS techniques with optimizations for streaming.
The paper tackles the challenge of streaming 3D Gaussian splatting (3DGS) volumetric scenes, which are bandwidth-intensive, by proposing GSStream, a system that integrates viewport prediction and bitrate adaptation modules, resulting in improved visual quality and network usage compared to existing systems.
Recently, the 3D Gaussian splatting (3DGS) technique for real-time radiance field rendering has revolutionized the field of volumetric scene representation, providing users with an immersive experience. But in return, it also poses a large amount of data volume, which is extremely bandwidth-intensive. Cutting-edge researchers have tried to introduce different approaches and construct multiple variants for 3DGS to obtain a more compact scene representation, but it is still challenging for real-time distribution. In this paper, we propose GSStream, a novel volumetric scene streaming system to support 3DGS data format. Specifically, GSStream integrates a collaborative viewport prediction module to better predict users' future behaviors by learning collaborative priors and historical priors from multiple users and users' viewport sequences and a deep reinforcement learning (DRL)-based bitrate adaptation module to tackle the state and action space variability challenge of the bitrate adaptation problem, achieving efficient volumetric scene delivery. Besides, we first build a user viewport trajectory dataset for volumetric scenes to support the training and streaming simulation. Extensive experiments prove that our proposed GSStream system outperforms existing representative volumetric scene streaming systems in visual quality and network usage. Demo video: https://youtu.be/3WEe8PN8yvA.