Find a Way Forward: a Language-Guided Semantic Map Navigator
This addresses the problem of language-guided navigation in 3D semantic maps for robotics or AI agents, representing an incremental improvement over existing methods.
The paper tackles the map-language navigation task by introducing an instruction-aware Path Proposal and Discrimination model (iPPD) that selects candidate paths and uses a discriminator to choose the best one, achieving gains of over 17% in navigation success and 0.18 in nDTW compared to a baseline in unseen environments.
In this paper, we introduce the map-language navigation task where an agent executes natural language instructions and moves to the target position based only on a given 3D semantic map. To tackle the task, we design the instruction-aware Path Proposal and Discrimination model (iPPD). Our approach leverages map information to provide instruction-aware path proposals, i.e., it selects all potential instruction-aligned candidate paths to reduce the solution space. Next, to represent the map observations along a path for a better modality alignment, a novel Path Feature Encoding scheme tailored for semantic maps is proposed. An attention-based Language Driven Discriminator is designed to evaluate path candidates and determine the best path as the final result. Our method can naturally avoid error accumulation compared with single-step greedy decision methods. Comparing to a single-step imitation learning approach, iPPD has performance gains above 17% on navigation success and 0.18 on path matching measurement nDTW in challenging unseen environments.