OMCL: Open-vocabulary Monte Carlo Localization
This addresses robot localization challenges for cross-modal scenarios, representing an incremental improvement over prior closed-set methods.
The paper tackles the problem of robust robot localization across different sensor modalities by extending Monte Carlo Localization with vision-language features, enabling open-vocabulary associations and natural language initialization. It demonstrates generalization across indoor (Matterport3D, Replica) and outdoor (SemanticKITTI) scenes.
Robust robot localization is an important prerequisite for navigation, but it becomes challenging when the map and robot measurements are obtained from different sensors. Prior methods are often tailored to specific environments, relying on closed-set semantics or fine-tuned features. In this work, we extend Monte Carlo Localization with vision-language features, allowing OMCL to robustly compute the likelihood of visual observations given a camera pose and a 3D map created from posed RGB-D images or aligned point clouds. These open-vocabulary features enable us to associate observations and map elements from different modalities, and to natively initialize global localization through natural language descriptions of nearby objects. We evaluate our approach using Matterport3D and Replica for indoor scenes and demonstrate generalization on SemanticKITTI for outdoor scenes.