CVAIApr 10

Learning Vision-Language-Action World Models for Autonomous Driving

arXiv:2604.0905989.12 citationsh-index: 6
AI Analysis

This work addresses the challenge of temporal dynamics and global consistency in autonomous driving, offering a novel integration that enhances foresight and interpretability, though it is incremental in combining existing paradigms.

The paper tackles the problem of autonomous driving by proposing VLA-World, a vision-language-action world model that unifies predictive imagination with reflective reasoning to improve foresight and safety, achieving higher performance than state-of-the-art baselines on planning and future-generation benchmarks.

Vision-Language-Action (VLA) models have recently achieved notable progress in end-to-end autonomous driving by integrating perception, reasoning, and control within a unified multimodal framework. However, they often lack explicit modeling of temporal dynamics and global world consistency, which limits their foresight and safety. In contrast, world models can simulate plausible future scenes but generally struggle to reason about or evaluate the imagined future they generate. In this work, we present VLA-World, a simple yet effective VLA world model that unifies predictive imagination with reflective reasoning to improve driving foresight. VLA-World first uses an action-derived feasible trajectory to guide the generation of the next-frame image, capturing rich spatial and temporal cues that describe how the surrounding environment evolves. The model then reasons over this self-generated future imagined frame to refine the predicted trajectory, achieving higher performance and better interpretability. To support this pipeline, we curate nuScenes-GR-20K, a generative reasoning dataset derived from nuScenes, and employ a three-stage training strategy that includes pretraining, supervised fine-tuning, and reinforcement learning. Extensive experiments demonstrate that VLA-World consistently surpasses state-of-the-art VLA and world-model baselines on both planning and future-generation benchmarks. Project page: https://vlaworld.github.io

Foundations

The foundational work for this paper's niche, ranked by how specifically the neighbourhood builds on it — not by global fame.

Your Notes