ClusterGNN: Cluster-based Coarse-to-Fine Graph Neural Network for Efficient Feature Matching
This addresses efficiency issues in computer vision tasks like feature matching, offering significant speed and memory improvements, though it is incremental as it builds on existing GNN methods.
The paper tackles the quadratic complexity of GNN-based visual feature matching by proposing ClusterGNN, which uses progressive clustering and a coarse-to-fine paradigm to reduce redundant connectivity, resulting in a 59.7% reduction in runtime and 58.4% reduction in memory consumption while maintaining competitive performance.
Graph Neural Networks (GNNs) with attention have been successfully applied for learning visual feature matching. However, current methods learn with complete graphs, resulting in a quadratic complexity in the number of features. Motivated by a prior observation that self- and cross- attention matrices converge to a sparse representation, we propose ClusterGNN, an attentional GNN architecture which operates on clusters for learning the feature matching task. Using a progressive clustering module we adaptively divide keypoints into different subgraphs to reduce redundant connectivity, and employ a coarse-to-fine paradigm for mitigating miss-classification within images. Our approach yields a 59.7% reduction in runtime and 58.4% reduction in memory consumption for dense detection, compared to current state-of-the-art GNN-based matching, while achieving a competitive performance on various computer vision tasks.