Co-Attention for Conditioned Image Matching
This addresses the problem of robust image matching for computer vision applications, offering a novel method that improves performance under hard conditions.
The paper tackles the problem of finding correspondences between image pairs under challenging conditions like large viewpoint changes by conditioning features on both images, achieving state-of-the-art or competitive results across tasks such as local matching and 3D reconstruction.
We propose a new approach to determine correspondences between image pairs in the wild under large changes in illumination, viewpoint, context, and material. While other approaches find correspondences between pairs of images by treating the images independently, we instead condition on both images to implicitly take account of the differences between them. To achieve this, we introduce (i) a spatial attention mechanism (a co-attention module, CoAM) for conditioning the learned features on both images, and (ii) a distinctiveness score used to choose the best matches at test time. CoAM can be added to standard architectures and trained using self-supervision or supervised data, and achieves a significant performance improvement under hard conditions, e.g. large viewpoint changes. We demonstrate that models using CoAM achieve state of the art or competitive results on a wide range of tasks: local matching, camera localization, 3D reconstruction, and image stylization.