OneDrive: Unified Multi-Paradigm Driving with Vision-Language-Action Models
For autonomous driving researchers, this work reduces architectural fragmentation by enabling a single VLM backbone to handle multi-task outputs, but it is incremental as it applies existing transformer techniques to a new domain.
OneDrive unifies heterogeneous decoding behaviors (autoregressive language, parallel detection, trajectory regression) into a single transformer decoder built on a pretrained VLM, achieving state-of-the-art autonomous driving performance: 0.28 L2 error and 0.18 collision rate on nuScenes open-loop, and 86.8 PDMS on NAVSIM closed-loop, with ~40% lower latency in efficient mode.
Vision-Language Models(VLMs) excel at autoregressive text generation, yet end-to-end autonomous driving requires multi-task learning with structured outputs and heterogeneous decoding behaviors, such as autoregressive language generation, parallel object detection and trajectory regression. To accommodate these differences, existing systems typically introduce separate or cascaded decoders, resulting in architectural fragmentation and limited backbone reuse. In this work, we present a unified autonomous driving framework built upon a pretrained VLM, where heterogeneous decoding behaviors are reconciled within a single transformer decoder. We demonstrate that pretrained VLM attention exhibits strong transferability beyond pure language modeling. By organizing visual and structured query tokens within a single causal decoder, structured queries can naturally condition on visual context through the original attention mechanism. Textual and structured outputs share a common attention backbone, enabling stable joint optimization across heterogeneous tasks. Trajectory planning is realized within the same causal LLM decoder by introducing structured trajectory queries. This unified formulation enables planning to share the pretrained attention backbone with images and perception tokens. Extensive experiments on end-to-end autonomous driving benchmarks demonstrate state-of-the-art performance, including 0.28 L2 and 0.18 collision rate on nuScenes open-loop evaluation and competitive results (86.8 PDMS) on NAVSIM closed-loop evaluation. The full model preserves multi-modal generation capability, while an efficient inference mode achieves approximately 40% lower latency. Code and models are available at https://github.com/Z1zyw/OneDrive