LaViPlan : Language-Guided Visual Path Planning with RLVR
This addresses the challenge of aligning language-based reasoning with low-level trajectory planning in autonomous driving, though it appears incremental as it builds on existing VLMs with RLVR fine-tuning.
The paper tackles the problem of out-of-distribution scenarios in autonomous driving by proposing LaViPlan, a framework that uses Reinforcement Learning with Verifiable Rewards to fine-tune Vision-Language Models for better planning performance, showing improvements across in-domain and out-of-domain datasets.
Out-of-distribution (OOD) scenarios in autonomous driving pose critical challenges, as planners often fail to generalize beyond their training experience, leading to unsafe or unexpected behavior. Vision-Language Models (VLMs) have shown promise in handling such scenarios by providing high-level scene understanding and user-aligned decisions. However, existing VLMs often exhibit a misalignment between their language-based reasoning and the low-level trajectories required for action-level planning. In this paper, we propose LaViPlan, a framework that leverages Reinforcement Learning with Verifiable Rewards (RLVR) to fine-tune VLMs using planning-oriented metrics. Experimental results show that LaViPlan improves planning performance across both in-domain and out-of-domain datasets. While linguistic fidelity slightly decreases after RLVR-based fine-tuning, qualitative evaluation indicates that the outputs remain coherent. We also conduct ablation studies to analyze the effects of sampling ratio and reasoning guidance, highlighting how these design choices influence performance. These findings demonstrate the potential of RLVR as a post-training paradigm for aligning language-guided reasoning with action-level planning in autonomous driving.