PCR-GS: COLMAP-Free 3D Gaussian Splatting via Pose Co-Regularizations
This addresses a bottleneck in 3D reconstruction for applications like robotics or AR/VR, but it is incremental as it builds on existing COLMAP-free 3D-GS techniques.
The paper tackles the problem of 3D Gaussian Splatting from unposed images or videos, which struggles with complex camera trajectories, by proposing PCR-GS, a method that uses camera pose co-regularization to improve scene modeling and pose estimation, achieving superior performance in experiments on real-world scenes.
COLMAP-free 3D Gaussian Splatting (3D-GS) has recently attracted increasing attention due to its remarkable performance in reconstructing high-quality 3D scenes from unposed images or videos. However, it often struggles to handle scenes with complex camera trajectories as featured by drastic rotation and translation across adjacent camera views, leading to degraded estimation of camera poses and further local minima in joint optimization of camera poses and 3D-GS. We propose PCR-GS, an innovative COLMAP-free 3DGS technique that achieves superior 3D scene modeling and camera pose estimation via camera pose co-regularization. PCR-GS achieves regularization from two perspectives. The first is feature reprojection regularization which extracts view-robust DINO features from adjacent camera views and aligns their semantic information for camera pose regularization. The second is wavelet-based frequency regularization which exploits discrepancy in high-frequency details to further optimize the rotation matrix in camera poses. Extensive experiments over multiple real-world scenes show that the proposed PCR-GS achieves superior pose-free 3D-GS scene modeling under dramatic changes of camera trajectories.