Map-free Visual Relocalization: Metric Pose Relative to a Single Image
This addresses the challenge of instant, metric-scaled relocalization for small places of interest like sculptures and murals, but it is incremental as it builds on existing methods like relative pose regression and feature matching.
The authors tackled the problem of visual relocalization using only a single reference image instead of a 3D map, proposing Map-free Relocalization and constructing a new dataset of 655 places with metric camera poses to benchmark it.
Can we relocalize in a scene represented by a single reference image? Standard visual relocalization requires hundreds of images and scale calibration to build a scene-specific 3D map. In contrast, we propose Map-free Relocalization, i.e., using only one photo of a scene to enable instant, metric scaled relocalization. Existing datasets are not suitable to benchmark map-free relocalization, due to their focus on large scenes or their limited variability. Thus, we have constructed a new dataset of 655 small places of interest, such as sculptures, murals and fountains, collected worldwide. Each place comes with a reference image to serve as a relocalization anchor, and dozens of query images with known, metric camera poses. The dataset features changing conditions, stark viewpoint changes, high variability across places, and queries with low to no visual overlap with the reference image. We identify two viable families of existing methods to provide baseline results: relative pose regression, and feature matching combined with single-image depth prediction. While these methods show reasonable performance on some favorable scenes in our dataset, map-free relocalization proves to be a challenge that requires new, innovative solutions.