Robust Scene Coordinate Regression via Geometrically-Consistent Global Descriptors
This work improves visual localization for robotics and AR/VR applications by enhancing robustness in large-scale environments, though it is incremental as it builds on existing learning-based methods.
The paper tackles the problem of visual localization by addressing limitations in existing global descriptors derived from geometric cues alone, proposing an aggregator module that learns descriptors consistent with both geometry and visual similarity, resulting in substantial localization gains on challenging benchmarks.
Recent learning-based visual localization methods use global descriptors to disambiguate visually similar places, but existing approaches often derive these descriptors from geometric cues alone (e.g., covisibility graphs), limiting their discriminative power and reducing robustness in the presence of noisy geometric constraints. We propose an aggregator module that learns global descriptors consistent with both geometrical structure and visual similarity, ensuring that images are close in descriptor space only when they are visually similar and spatially connected. This corrects erroneous associations caused by unreliable overlap scores. Using a batch-mining strategy based solely on the overlap scores and a modified contrastive loss, our method trains without manual place labels and generalizes across diverse environments. Experiments on challenging benchmarks show substantial localization gains in large-scale environments while preserving computational and memory efficiency. Code is available at https://github.com/sontung/robust_scr.