Bridge 2D-3D: Uncertainty-aware Hierarchical Registration Network with Domain Alignment
This work improves registration accuracy for computer vision applications, but it appears incremental as it builds on existing coarse-to-fine pipelines.
The paper tackles the problem of image-to-point cloud registration by addressing incorrect patch matching and domain gaps, achieving state-of-the-art results on RGB-D Scene V2 and 7-Scenes benchmarks.
The method for image-to-point cloud registration typically determines the rigid transformation using a coarse-to-fine pipeline. However, directly and uniformly matching image patches with point cloud patches may lead to focusing on incorrect noise patches during matching while ignoring key ones. Moreover, due to the significant differences between image and point cloud modalities, it may be challenging to bridge the domain gap without specific improvements in design. To address the above issues, we innovatively propose the Uncertainty-aware Hierarchical Matching Module (UHMM) and the Adversarial Modal Alignment Module (AMAM). Within the UHMM, we model the uncertainty of critical information in image patches and facilitate multi-level fusion interactions between image and point cloud features. In the AMAM, we design an adversarial approach to reduce the domain gap between image and point cloud. Extensive experiments and ablation studies on RGB-D Scene V2 and 7-Scenes benchmarks demonstrate the superiority of our method, making it a state-of-the-art approach for image-to-point cloud registration tasks.