CRISTAL: Real-time Camera Registration in Static LiDAR Scans using Neural Rendering
This addresses the problem of accurate camera localization for robotics and XR applications, offering a solution to drift and scale issues, but it is incremental as it builds on neural rendering and point cloud techniques.
The paper tackles camera localization in robotics and XR by introducing a real-time method that registers a camera within a pre-captured colored LiDAR point cloud, achieving drift-free tracking with correct metric scale and outperforming existing SLAM pipelines on the ScanNet++ dataset.
Accurate camera localization is crucial for robotics and Extended Reality (XR), enabling reliable navigation and alignment of virtual and real content. Existing visual methods often suffer from drift, scale ambiguity, and depend on fiducials or loop closure. This work introduces a real-time method for localizing a camera within a pre-captured, highly accurate colored LiDAR point cloud. By rendering synthetic views from this cloud, 2D-3D correspondences are established between live frames and the point cloud. A neural rendering technique narrows the domain gap between synthetic and real images, reducing occlusion and background artifacts to improve feature matching. The result is drift-free camera tracking with correct metric scale in the global LiDAR coordinate system. Two real-time variants are presented: Online Render and Match, and Prebuild and Localize. We demonstrate improved results on the ScanNet++ dataset and outperform existing SLAM pipelines.