Pose-Free Omnidirectional Gaussian Splatting for 360-Degree Videos with Consistent Depth Priors
This work addresses the challenge of efficient 3D scene representation for 360-degree videos, offering a pose-free solution that improves novel view synthesis, though it appears incremental as it builds upon existing Gaussian Splatting techniques.
The paper tackles the problem of reconstructing 3D scenes from unposed omnidirectional videos without relying on slow structure-from-motion methods, achieving significant outperformance over existing pose-free and pose-aware methods on real-world and synthetic 360-degree videos.
Omnidirectional 3D Gaussian Splatting with panoramas is a key technique for 3D scene representation, and existing methods typically rely on slow SfM to provide camera poses and sparse points priors. In this work, we propose a pose-free omnidirectional 3DGS method, named PFGS360, that reconstructs 3D Gaussians from unposed omnidirectional videos. To achieve accurate camera pose estimation, we first construct a spherical consistency-aware pose estimation module, which recovers poses by establishing consistent 2D-3D correspondences between the reconstructed Gaussians and the unposed images using Gaussians' internal depth priors. Besides, to enhance the fidelity of novel view synthesis, we introduce a depth-inlier-aware densification module to extract depth inliers and Gaussian outliers with consistent monocular depth priors, enabling efficient Gaussian densification and achieving photorealistic novel view synthesis. The experiments show significant outperformance over existing pose-free and pose-aware 3DGS methods on both real-world and synthetic 360-degree videos. Code is available at https://github.com/zcq15/PFGS360.