Endowing Embodied Agents with Spatial Reasoning Capabilities for Vision-and-Language Navigation
This addresses the challenge of reliable spatial reasoning for mobile robots in real-world navigation tasks, representing a novel method rather than an incremental improvement.
The paper tackles the problem of spatial hallucinations in embodied Vision-and-Language Navigation (VLN) when transferring from simulated to real-world environments, proposing BrainNav, a bio-inspired framework that integrates dual-map and dual-orientation strategies, and it outperforms existing SOTA methods in zero-shot real-world tests without fine-tuning.
Enhancing the spatial perception capabilities of mobile robots is crucial for achieving embodied Vision-and-Language Navigation (VLN). Although significant progress has been made in simulated environments, directly transferring these capabilities to real-world scenarios often results in severe hallucination phenomena, causing robots to lose effective spatial awareness. To address this issue, we propose BrainNav, a bio-inspired spatial cognitive navigation framework inspired by biological spatial cognition theories and cognitive map theory. BrainNav integrates dual-map (coordinate map and topological map) and dual-orientation (relative orientation and absolute orientation) strategies, enabling real-time navigation through dynamic scene capture and path planning. Its five core modules-Hippocampal Memory Hub, Visual Cortex Perception Engine, Parietal Spatial Constructor, Prefrontal Decision Center, and Cerebellar Motion Execution Unit-mimic biological cognitive functions to reduce spatial hallucinations and enhance adaptability. Validated in a zero-shot real-world lab environment using the Limo Pro robot, BrainNav, compatible with GPT-4, outperforms existing State-of-the-Art (SOTA) Vision-and-Language Navigation in Continuous Environments (VLN-CE) methods without fine-tuning.