LiftFeat: 3D Geometry-Aware Local Feature Matching
This work addresses a key problem in robotics applications like SLAM and visual localization by improving feature matching robustness, though it appears incremental as it builds on existing methods by adding 3D geometry awareness.
The paper tackles the challenge of robust local feature matching in difficult visual conditions like lighting changes and low texture by proposing LiftFeat, a lightweight network that integrates 3D geometric features to enhance 2D descriptors, resulting in improved performance on tasks such as relative pose estimation and visual localization compared to other lightweight state-of-the-art methods.
Robust and efficient local feature matching plays a crucial role in applications such as SLAM and visual localization for robotics. Despite great progress, it is still very challenging to extract robust and discriminative visual features in scenarios with drastic lighting changes, low texture areas, or repetitive patterns. In this paper, we propose a new lightweight network called \textit{LiftFeat}, which lifts the robustness of raw descriptor by aggregating 3D geometric feature. Specifically, we first adopt a pre-trained monocular depth estimation model to generate pseudo surface normal label, supervising the extraction of 3D geometric feature in terms of predicted surface normal. We then design a 3D geometry-aware feature lifting module to fuse surface normal feature with raw 2D descriptor feature. Integrating such 3D geometric feature enhances the discriminative ability of 2D feature description in extreme conditions. Extensive experimental results on relative pose estimation, homography estimation, and visual localization tasks, demonstrate that our LiftFeat outperforms some lightweight state-of-the-art methods. Code will be released at : https://github.com/lyp-deeplearning/LiftFeat.