VAMOS: A Hierarchical Vision-Language-Action Model for Capability-Modulated and Steerable Navigation
This work addresses the problem of cross-embodied navigation for robots, enabling deployment across physically distinct types like legged and wheeled robots, though it is incremental in building on hierarchical and simulation-based approaches.
The paper tackles the challenge of learning robot navigation policies that generalize across environments while adapting to specific robot capabilities, proposing VAMOS, a hierarchical vision-language-action model that decouples semantic planning from embodiment grounding, resulting in higher success rates in real-world indoor and outdoor navigation compared to state-of-the-art methods, with a 3X improvement in reliability by rejecting infeasible plans.
A fundamental challenge in robot navigation lies in learning policies that generalize across diverse environments while conforming to the unique physical constraints and capabilities of a specific embodiment (e.g., quadrupeds can walk up stairs, but rovers cannot). We propose VAMOS, a hierarchical VLA that decouples semantic planning from embodiment grounding: a generalist planner learns from diverse, open-world data, while a specialist affordance model learns the robot's physical constraints and capabilities in safe, low-cost simulation. We enabled this separation by carefully designing an interface that lets a high-level planner propose candidate paths directly in image space that the affordance model then evaluates and re-ranks. Our real-world experiments show that VAMOS achieves higher success rates in both indoor and complex outdoor navigation than state-of-the-art model-based and end-to-end learning methods. We also show that our hierarchical design enables cross-embodied navigation across legged and wheeled robots and is easily steerable using natural language. Real-world ablations confirm that the specialist model is key to embodiment grounding, enabling a single high-level planner to be deployed across physically distinct wheeled and legged robots. Finally, this model significantly enhances single-robot reliability, achieving 3X higher success rates by rejecting physically infeasible plans. Website: https://vamos-vla.github.io/