Efficient Dense Matching for Enhanced Gaussian Splatting Using AV1 Motion Vectors
For practitioners of 3D Gaussian Splatting, this work offers a more efficient initialization method that improves reconstruction quality and training speed, though it is incremental as it applies existing video codec techniques to a known bottleneck.
This paper introduces an AV1-based feature detection and matching pipeline that reduces SfM processing overhead and produces up to eight times denser point clouds than classical SfM, leading to a 9-point increase in VMAF and 63% reduction in 3DGS training time to reach baseline quality.
3D Gaussian Splatting (3DGS) has emerged as a prominent framework for real-time, photorealistic scene reconstruction, offering significant speed-ups over Neural Radiance Fields (NeRF). However, the fidelity of 3DGS representations remains heavily dependent on the quality of the initial point cloud. While standard Structure-from-Motion (SfM) pipelines using COLMAP provide adequate initialisation, they often suffer from high computational costs and sparsity in textureless regions, which degrades subsequent reconstruction accuracy and convergence speed. In this work, we introduce an AV1-based feature detection and matching pipeline that significantly reduces SfM processing overhead. By leveraging motion vectors inherent to the AV1 video codec, we bypass computationally expensive exhaustive matching while maintaining geometric robustness. Our pipeline produces substantially denser point clouds, with up to eight times as many points as classical SfM. We demonstrate that this enhanced initialisation directly improves 3DGS performance, yielding an 9-point increase in VMAF and a 63% average reduction in training time required to reach baseline quality. The project page: https://sigmedia.tv/AV1-3DGS.github.io/