Vision-and-Language Navigation with Analogical Textual Descriptions in LLMs
This work addresses the challenge of better contextual understanding in embodied AI for navigation, though it appears incremental as it builds on existing LLM-based VLN methods.
The paper tackles the problem of improving zero-shot Vision-and-Language Navigation (VLN) by incorporating textual descriptions from multiple perspectives to enhance analogical reasoning, resulting in significant performance improvements on the R2R dataset.
Integrating large language models (LLMs) into embodied AI models is becoming increasingly prevalent. However, existing zero-shot LLM-based Vision-and-Language Navigation (VLN) agents either encode images as textual scene descriptions, potentially oversimplifying visual details, or process raw image inputs, which can fail to capture abstract semantics required for high-level reasoning. In this paper, we improve the navigation agent's contextual understanding by incorporating textual descriptions from multiple perspectives that facilitate analogical reasoning across images. By leveraging text-based analogical reasoning, the agent enhances its global scene understanding and spatial reasoning, leading to more accurate action decisions. We evaluate our approach on the R2R dataset, where our experiments demonstrate significant improvements in navigation performance.