CT-NeRF: Incremental Optimizing Neural Radiance Field and Poses with Complex Trajectory
This addresses a bottleneck in NeRF-based reconstruction for applications like robotics or AR/VR, but it is incremental as it builds on prior pose optimization methods.
The paper tackles the problem of 3D scene reconstruction with neural radiance fields (NeRF) when camera poses are unknown and trajectories are complex, proposing CT-NeRF to recover poses and scene structure from RGB images, and it outperforms existing methods on real-world datasets.
Neural radiance field (NeRF) has achieved impressive results in high-quality 3D scene reconstruction. However, NeRF heavily relies on precise camera poses. While recent works like BARF have introduced camera pose optimization within NeRF, their applicability is limited to simple trajectory scenes. Existing methods struggle while tackling complex trajectories involving large rotations. To address this limitation, we propose CT-NeRF, an incremental reconstruction optimization pipeline using only RGB images without pose and depth input. In this pipeline, we first propose a local-global bundle adjustment under a pose graph connecting neighboring frames to enforce the consistency between poses to escape the local minima caused by only pose consistency with the scene structure. Further, we instantiate the consistency between poses as a reprojected geometric image distance constraint resulting from pixel-level correspondences between input image pairs. Through the incremental reconstruction, CT-NeRF enables the recovery of both camera poses and scene structure and is capable of handling scenes with complex trajectories. We evaluate the performance of CT-NeRF on two real-world datasets, NeRFBuster and Free-Dataset, which feature complex trajectories. Results show CT-NeRF outperforms existing methods in novel view synthesis and pose estimation accuracy.