Long-term Visual Localization with Mobile Sensors
This addresses the challenge of long-term visual localization for mobile devices in dynamic outdoor settings, though it is incremental by integrating sensor data into existing frameworks.
The paper tackles the problem of camera localization in temporally-varying outdoor environments by leveraging mobile sensors (GPS, compass, gravity) to provide initial poses and constraints, reducing search space and enabling efficient 2D-3D matching, resulting in demonstrated effectiveness on a new dataset with significant appearance variations.
Despite the remarkable advances in image matching and pose estimation, image-based localization of a camera in a temporally-varying outdoor environment is still a challenging problem due to huge appearance disparity between query and reference images caused by illumination, seasonal and structural changes. In this work, we propose to leverage additional sensors on a mobile phone, mainly GPS, compass, and gravity sensor, to solve this challenging problem. We show that these mobile sensors provide decent initial poses and effective constraints to reduce the searching space in image matching and final pose estimation. With the initial pose, we are also able to devise a direct 2D-3D matching network to efficiently establish 2D-3D correspondences instead of tedious 2D-2D matching in existing systems. As no public dataset exists for the studied problem, we collect a new dataset that provides a variety of mobile sensor data and significant scene appearance variations, and develop a system to acquire ground-truth poses for query images. We benchmark our method as well as several state-of-the-art baselines and demonstrate the effectiveness of the proposed approach. The code and dataset will be released publicly.