MonoDream: Monocular Vision-Language Navigation with Panoramic Dreaming
This addresses the challenge of costly or inaccessible panoramic sensors in real-world VLN deployments, though it is incremental as it builds on existing monocular VLA models.
The paper tackles the problem of Vision-Language Navigation (VLN) by proposing MonoDream, a lightweight framework that enables monocular agents to learn a Unified Navigation Representation, improving performance and narrowing the gap with panoramic-based agents on multiple benchmarks.
Vision-Language Navigation (VLN) tasks often leverage panoramic RGB and depth inputs to provide rich spatial cues for action planning, but these sensors can be costly or less accessible in real-world deployments. Recent approaches based on Vision-Language Action (VLA) models achieve strong results with monocular input, yet they still lag behind methods using panoramic RGB-D information. We present MonoDream, a lightweight VLA framework that enables monocular agents to learn a Unified Navigation Representation (UNR). This shared feature representation jointly aligns navigation-relevant visual semantics (e.g., global layout, depth, and future cues) and language-grounded action intent, enabling more reliable action prediction. MonoDream further introduces Latent Panoramic Dreaming (LPD) tasks to supervise the UNR, which train the model to predict latent features of panoramic RGB and depth observations at both current and future steps based on only monocular input. Experiments on multiple VLN benchmarks show that MonoDream consistently improves monocular navigation performance and significantly narrows the gap with panoramic-based agents.