Guide Local Feature Matching by Overlap Estimation
This work addresses a key problem in computer vision for applications like 3D reconstruction and image stitching, but it is incremental as it builds on existing local feature matching pipelines.
The paper tackles the challenge of local image feature matching under large appearance, viewpoint, and distance changes by introducing OETR, an overlap estimation method that constrains matching to commonly visible regions, resulting in substantial performance boosts for state-of-the-art methods, especially for image pairs with small shared regions.
Local image feature matching under large appearance, viewpoint, and distance changes is challenging yet important. Conventional methods detect and match tentative local features across the whole images, with heuristic consistency checks to guarantee reliable matches. In this paper, we introduce a novel Overlap Estimation method conditioned on image pairs with TRansformer, named OETR, to constrain local feature matching in the commonly visible region. OETR performs overlap estimation in a two-step process of feature correlation and then overlap regression. As a preprocessing module, OETR can be plugged into any existing local feature detection and matching pipeline, to mitigate potential view angle or scale variance. Intensive experiments show that OETR can boost state-of-the-art local feature matching performance substantially, especially for image pairs with small shared regions. The code will be publicly available at https://github.com/AbyssGaze/OETR.