Targetless LiDAR-Camera Calibration with Neural Gaussian Splatting
This addresses the need for practical, drift-resistant calibration in multi-sensor systems, though it is incremental as it builds on neural scene representations.
The paper tackles the problem of LiDAR-camera calibration without physical targets by jointly optimizing sensor poses with a neural Gaussian-based scene representation, achieving robust performance that outperforms existing targetless methods on datasets like KITTI-360 and surpasses provided calibrations in rendering quality.
Accurate LiDAR-camera calibration is crucial for multi-sensor systems. However, traditional methods often rely on physical targets, which are impractical for real-world deployment. Moreover, even carefully calibrated extrinsics can degrade over time due to sensor drift or external disturbances, necessitating periodic recalibration. To address these challenges, we present a Targetless LiDAR-Camera Calibration (TLC-Calib) that jointly optimizes sensor poses with a neural Gaussian-based scene representation. Reliable LiDAR points are frozen as anchor Gaussians to preserve global structure, while auxiliary Gaussians prevent local overfitting under noisy initialization. Our fully differentiable pipeline with photometric and geometric regularization achieves robust and generalizable calibration, consistently outperforming existing targetless methods on KITTI-360, Waymo, and FAST-LIVO2, and surpassing even the provided calibrations in rendering quality.