Quantity-Aware Coarse-to-Fine Correspondence for Image-to-Point Cloud Registration
This addresses the challenge of locating 3D objects from 2D observations for applications like autonomous driving, though it appears incremental as it builds on existing correspondence and RANSAC methods.
The paper tackles the problem of image-to-point cloud registration by proposing a framework that captures quantity-aware correspondences between local point sets and pixel patches, then refines them at point and pixel levels. Experiments on KITTI Odometry and NuScenes datasets show the method outperforms state-of-the-art approaches.
Image-to-point cloud registration aims to determine the relative camera pose between an RGB image and a reference point cloud, serving as a general solution for locating 3D objects from 2D observations. Matching individual points with pixels can be inherently ambiguous due to modality gaps. To address this challenge, we propose a framework to capture quantity-aware correspondences between local point sets and pixel patches and refine the results at both the point and pixel levels. This framework aligns the high-level semantics of point sets and pixel patches to improve the matching accuracy. On a coarse scale, the set-to-patch correspondence is expected to be influenced by the quantity of 3D points. To achieve this, a novel supervision strategy is proposed to adaptively quantify the degrees of correlation as continuous values. On a finer scale, point-to-pixel correspondences are refined from a smaller search space through a well-designed scheme, which incorporates both resampling and quantity-aware priors. Particularly, a confidence sorting strategy is proposed to proportionally select better correspondences at the final stage. Leveraging the advantages of high-quality correspondences, the problem is successfully resolved using an efficient Perspective-n-Point solver within the framework of random sample consensus (RANSAC). Extensive experiments on the KITTI Odometry and NuScenes datasets demonstrate the superiority of our method over the state-of-the-art methods.