SuperGlue: Learning Feature Matching with Graph Neural Networks
This addresses the feature matching problem in computer vision for applications like SfM or SLAM, with incremental improvements over existing learned approaches.
The paper tackles the problem of matching local features between images for pose estimation by introducing SuperGlue, a neural network that jointly finds correspondences and rejects non-matchable points, achieving state-of-the-art results in challenging real-world indoor and outdoor environments.
This paper introduces SuperGlue, a neural network that matches two sets of local features by jointly finding correspondences and rejecting non-matchable points. Assignments are estimated by solving a differentiable optimal transport problem, whose costs are predicted by a graph neural network. We introduce a flexible context aggregation mechanism based on attention, enabling SuperGlue to reason about the underlying 3D scene and feature assignments jointly. Compared to traditional, hand-designed heuristics, our technique learns priors over geometric transformations and regularities of the 3D world through end-to-end training from image pairs. SuperGlue outperforms other learned approaches and achieves state-of-the-art results on the task of pose estimation in challenging real-world indoor and outdoor environments. The proposed method performs matching in real-time on a modern GPU and can be readily integrated into modern SfM or SLAM systems. The code and trained weights are publicly available at https://github.com/magicleap/SuperGluePretrainedNetwork.