CrossMaps: Confidence-Aware Open-Vocabulary Semantic Mapping for Rover Navigation
This work addresses the need for robust, real-time semantic mapping in autonomous rover navigation, but it is an incremental improvement over VLMaps-style approaches.
CrossMaps introduces a real-time, confidence-aware open-vocabulary semantic mapping pipeline for rover navigation, integrating multi-scale CLIP embeddings with a dual-memory architecture to produce language-queryable maps from RGB-D data. The system runs on a Jetson Orin-powered UGV and enables natural language queries for navigation guidance.
Rovers rely on perception to maintain spatial maps that encode both objects and sensor quality (e.g., range reliability, lighting artifacts, data density), guiding data fusion, embedding updates, and navigation under partial observability. To study these coupled perception-navigation processes, we present CrossMaps, a real-time confidence-aware open-vocabulary semantic mapping pipeline that constructs language-queryable maps from RGB-D data. Building on VLMaps-style approaches, CrossMaps integrates multi-scale CLIP embeddings with confidence-aware fusion and a dual-memory architecture consisting of Short-Term Memory (STM) and Long-Term Memory (LTM). The STM aggregates noisy visual observations using geometric, semantic, and temporal confidence cues, while confident and coherent cells are promoted to the LTM as persistent semantic landmarks. Designed for deployment with a Jetson Orin-powered UGV alongside SLAM, CrossMaps runs in real time and produces semantic heatmaps that can be queried with natural language to guide rover navigation.