Metis: A Generalizable and Efficient World-Action Model for Autonomous Driving and Urban Navigation
This work addresses the latency and generalization issues in world-action models for autonomous driving, offering a more efficient and generalizable framework for urban navigation.
Metis introduces a decoupled world-action model for autonomous driving that separates video generation from action prediction, achieving state-of-the-art performance on NAVSIM and CityWalker benchmarks while reducing inference latency by bypassing video generation at test time.
World action models~(WAMs) have shown great promise for autonomous driving and urban navigation. Built upon Vision-Language-Action models or video generation models, existing approaches suffer key limitations: (1) High inference latency due to future observation prediction at test time, and (2) tightly coupled video and action modeling leading to representational mismatch and degraded generalization. To address both issues, we propose Metis, an end-to-end WAM framework that decouples video generation and action prediction. Specifically, Metis employs a Mixture-of-Transformers architecture with dedicated experts for video generation and action prediction, preserving the intrinsic distributional properties of each task. To enhance efficiency, we introduce an asymmetric attention mask that enables joint training of both experts while allowing the action model to bypass explicit video generation during inference. This design ensures training-inference consistency and significantly reduces computational costs without compromising planning performance. Extensive experiments demonstrate state-of-the-art performance on the NAVSIM navhard and navtest benchmarks and the CityWalker navigation benchmark, validating both the generalizability and efficiency across diverse tasks. Real-robot deployments further confirm the practical feasibility of our approach.