LoD-Structured 3D Gaussian Splatting for Streaming Video Reconstruction
This addresses the challenge of streaming photorealistic 3D video with sparse inputs and bandwidth constraints, representing an incremental improvement over existing 3D Gaussian splatting methods.
The paper tackled the problem of real-time streaming free-viewpoint video reconstruction by proposing StreamLoD-GS, a framework that integrates LoD-structured 3D Gaussian splatting, motion partitioning, and quantized refinement, achieving competitive or state-of-the-art performance in quality, efficiency, and storage.
Free-Viewpoint Video (FVV) reconstruction enables photorealistic and interactive 3D scene visualization; however, real-time streaming is often bottlenecked by sparse-view inputs, prohibitive training costs, and bandwidth constraints. While recent 3D Gaussian Splatting (3DGS) has advanced FVV due to its superior rendering speed, Streaming Free-Viewpoint Video (SFVV) introduces additional demands for rapid optimization, high-fidelity reconstruction under sparse constraints, and minimal storage footprints. To bridge this gap, we propose StreamLoD-GS, an LoD-based Gaussian Splatting framework designed specifically for SFVV. Our approach integrates three core innovations: 1) an Anchor- and Octree-based LoD-structured 3DGS with a hierarchical Gaussian dropout technique to ensure efficient and stable optimization while maintaining high-quality rendering; 2) a GMM-based motion partitioning mechanism that separates dynamic and static content, refining dynamic regions while preserving background stability; and 3) a quantized residual refinement framework that significantly reduces storage requirements without compromising visual fidelity. Extensive experiments demonstrate that StreamLoD-GS achieves competitive or state-of-the-art performance in terms of quality, efficiency, and storage.