A-SCoRe: Attention-based Scene Coordinate Regression for wide-ranging scenarios
This work addresses visual localization for robotic and vision systems by offering a more efficient and versatile alternative to existing methods, though it appears incremental as it builds on scene coordinate regression with a new architecture.
The paper tackles the problem of visual localization by proposing A-SCoRe, an attention-based scene coordinate regression method that reduces storage and compute burdens compared to feature matching approaches, achieving comparable performance to state-of-the-art methods on multiple benchmarks while being lightweight and flexible.
Visual localization is considered to be one of the crucial parts in many robotic and vision systems. While state-of-the art methods that relies on feature matching have proven to be accurate for visual localization, its requirements for storage and compute are burdens. Scene coordinate regression (SCR) is an alternative approach that remove the barrier for storage by learning to map 2D pixels to 3D scene coordinates. Most popular SCR use Convolutional Neural Network (CNN) to extract 2D descriptor, which we would argue that it miss the spatial relationship between pixels. Inspired by the success of vision transformer architecture, we present a new SCR architecture, called A-ScoRe, an Attention-based model which leverage attention on descriptor map level to produce meaningful and high-semantic 2D descriptors. Since the operation is performed on descriptor map, our model can work with multiple data modality whether it is a dense or sparse from depth-map, SLAM to Structure-from-Motion (SfM). This versatility allows A-SCoRe to operate in different kind of environments, conditions and achieve the level of flexibility that is important for mobile robots. Results show our methods achieve comparable performance with State-of-the-art methods on multiple benchmark while being light-weighted and much more flexible. Code and pre-trained models are public in our repository: https://github.com/ais-lab/A-SCoRe.