BIT-Nav: Brain-Inspired Trajectory Memory for Embodied Navigation
For embodied navigation agents, this work provides a token-efficient way to inject structured motion history into VLMs, improving long-horizon reasoning without scaling frame count.
BIT-Nav addresses the bottleneck in VLM-based embodied navigation where sparse frame selection fails to capture structured behavioral signals for long-horizon reasoning. By augmenting frozen VLMs with a compact learned trajectory memory (a single token) derived from action and pose sequences, it achieves a 12% improvement in success rate and 15% in SPL over the baseline on the VLN-CE benchmark.
Vision-Language Models (VLMs) for embodied navigation rely on selecting a fixed number of frames from a growing trajectory history. As episodes extend, this selection grows increasingly sparse, yet prior work shows no accuracy gain when scaling from 8 to 64 frames, suggesting the bottleneck is not frame quantity but the representation itself. Sparse frame selection cannot capture the structured behavioral signal that long-horizon reasoning requires: turning patterns, cumulative displacement, and path topology. We introduce BIT-Nav (Brain-Inspired Trajectory Memory for Navigation), a framework that augments frozen VLM navigation pipelines with a compact learned trajectory memory. Motivated by hippocampal path integration, where spatial experience is compressed into structured episodic traces rather than stored as raw sensory replay, BIT-Nav trains a Bi-GRU encoder over action and relative pose sequences via a multi-positive InfoNCE contrastive objective on trajectory prefixes sharing the same behavioral intent. The resulting embedding is projected into the VLM token space via a lightweight MLP and injected as a single memory token at each decision step, conditioning the model on structured motion history at constant token cost regardless of episode length