CompSplat: Compression-aware 3D Gaussian Splatting for Real-world Video
This addresses the problem of compression artifacts degrading 3D reconstruction quality for applications like cultural heritage preservation and digital twins, representing an incremental improvement over existing compression-aware approaches.
The paper tackles the problem of novel view synthesis from real-world videos with compression artifacts by proposing CompSplat, a compression-aware training framework that models frame-wise compression characteristics to mitigate inter-frame inconsistency and geometric errors. The result is state-of-the-art rendering quality and pose accuracy on benchmarks like Tanks and Temples, Free, and Hike, significantly outperforming recent methods under severe compression.
High-quality novel view synthesis (NVS) from real-world videos is crucial for applications such as cultural heritage preservation, digital twins, and immersive media. However, real-world videos typically contain long sequences with irregular camera trajectories and unknown poses, leading to pose drift, feature misalignment, and geometric distortion during reconstruction. Moreover, lossy compression amplifies these issues by introducing inconsistencies that gradually degrade geometry and rendering quality. While recent studies have addressed either long-sequence NVS or unposed reconstruction, compression-aware approaches still focus on specific artifacts or limited scenarios, leaving diverse compression patterns in long videos insufficiently explored. In this paper, we propose CompSplat, a compression-aware training framework that explicitly models frame-wise compression characteristics to mitigate inter-frame inconsistency and accumulated geometric errors. CompSplat incorporates compression-aware frame weighting and an adaptive pruning strategy to enhance robustness and geometric consistency, particularly under heavy compression. Extensive experiments on challenging benchmarks, including Tanks and Temples, Free, and Hike, demonstrate that CompSplat achieves state-of-the-art rendering quality and pose accuracy, significantly surpassing most recent state-of-the-art NVS approaches under severe compression conditions.