MeshLoc: Mesh-Based Visual Localization
This addresses the need for more adaptable scene representations in applications like autonomous robots and augmented reality, though it is incremental as it builds on existing mesh-based approaches.
The paper tackles the problem of visual localization for camera pose estimation by proposing a flexible alternative based on dense 3D meshes that avoids expensive feature matching between database images, achieving state-of-the-art results and showing competitive performance even with simple renderings without color or texture.
Visual localization, i.e., the problem of camera pose estimation, is a central component of applications such as autonomous robots and augmented reality systems. A dominant approach in the literature, shown to scale to large scenes and to handle complex illumination and seasonal changes, is based on local features extracted from images. The scene representation is a sparse Structure-from-Motion point cloud that is tied to a specific local feature. Switching to another feature type requires an expensive feature matching step between the database images used to construct the point cloud. In this work, we thus explore a more flexible alternative based on dense 3D meshes that does not require features matching between database images to build the scene representation. We show that this approach can achieve state-of-the-art results. We further show that surprisingly competitive results can be obtained when extracting features on renderings of these meshes, without any neural rendering stage, and even when rendering raw scene geometry without color or texture. Our results show that dense 3D model-based representations are a promising alternative to existing representations and point to interesting and challenging directions for future research.