Learning-Based Dimensionality Reduction for Computing Compact and Effective Local Feature Descriptors
This work addresses memory and computational efficiency issues for computer vision and robotics applications, offering an incremental improvement over existing dimensionality reduction methods.
The paper tackles the problem of high-dimensional local feature descriptors in computer vision by using multi-layer perceptrons (MLPs) to reduce dimensionality, achieving better performance than PCA and improving downstream tasks like visual localization and image matching, especially for hand-crafted descriptors.
A distinctive representation of image patches in form of features is a key component of many computer vision and robotics tasks, such as image matching, image retrieval, and visual localization. State-of-the-art descriptors, from hand-crafted descriptors such as SIFT to learned ones such as HardNet, are usually high dimensional; 128 dimensions or even more. The higher the dimensionality, the larger the memory consumption and computational time for approaches using such descriptors. In this paper, we investigate multi-layer perceptrons (MLPs) to extract low-dimensional but high-quality descriptors. We thoroughly analyze our method in unsupervised, self-supervised, and supervised settings, and evaluate the dimensionality reduction results on four representative descriptors. We consider different applications, including visual localization, patch verification, image matching and retrieval. The experiments show that our lightweight MLPs achieve better dimensionality reduction than PCA. The lower-dimensional descriptors generated by our approach outperform the original higher-dimensional descriptors in downstream tasks, especially for the hand-crafted ones. The code will be available at https://github.com/PRBonn/descriptor-dr.