Breaking Modality Disparity: Harmonized Representation for Infrared and Visible Image Registration
This addresses the time-consuming manual calibration needed for infrared-visible image registration, benefiting applications in surveillance or autonomous systems, though it appears incremental as it builds on existing registration techniques.
The paper tackles the problem of registering infrared and visible images, which is crucial for accurate scene perception in multi-modality sensing, by proposing a scene-adaptive method that uses an invertible translation process and hierarchical framework, achieving state-of-the-art results validated through extensive experiments.
Since the differences in viewing range, resolution and relative position, the multi-modality sensing module composed of infrared and visible cameras needs to be registered so as to have more accurate scene perception. In practice, manual calibration-based registration is the most widely used process, and it is regularly calibrated to maintain accuracy, which is time-consuming and labor-intensive. To cope with these problems, we propose a scene-adaptive infrared and visible image registration. Specifically, in regard of the discrepancy between multi-modality images, an invertible translation process is developed to establish a modality-invariant domain, which comprehensively embraces the feature intensity and distribution of both infrared and visible modalities. We employ homography to simulate the deformation between different planes and develop a hierarchical framework to rectify the deformation inferred from the proposed latent representation in a coarse-to-fine manner. For that, the advanced perception ability coupled with the residual estimation conducive to the regression of sparse offsets, and the alternate correlation search facilitates a more accurate correspondence matching. Moreover, we propose the first ground truth available misaligned infrared and visible image dataset, involving three synthetic sets and one real-world set. Extensive experiments validate the effectiveness of the proposed method against the state-of-the-arts, advancing the subsequent applications.