Weatherproofing Retrieval for Localization with Generative AI and Geometric Consistency
This work addresses a critical bottleneck in visual localization for applications like autonomous navigation, though it is incremental as it builds on existing retrieval methods.
The paper tackles the problem of visual localization accuracy degradation due to varying conditions like weather and time of day by improving the image retrieval step through synthetic data generation and geometric consistency training, resulting in large improvements on challenging datasets.
State-of-the-art visual localization approaches generally rely on a first image retrieval step whose role is crucial. Yet, retrieval often struggles when facing varying conditions, due to e.g. weather or time of day, with dramatic consequences on the visual localization accuracy. In this paper, we improve this retrieval step and tailor it to the final localization task. Among the several changes we advocate for, we propose to synthesize variants of the training set images, obtained from generative text-to-image models, in order to automatically expand the training set towards a number of nameable variations that particularly hurt visual localization. After expanding the training set, we propose a training approach that leverages the specificities and the underlying geometry of this mix of real and synthetic images. We experimentally show that those changes translate into large improvements for the most challenging visual localization datasets. Project page: https://europe.naverlabs.com/ret4loc