AdaDrive: Self-Adaptive Slow-Fast System for Language-Grounded Autonomous Driving
This work addresses the problem of balancing high-level reasoning with real-time efficiency for autonomous driving systems, representing an incremental improvement over existing methods.
The paper tackles the challenge of integrating Large Language Models (LLMs) into autonomous driving by proposing AdaDrive, a self-adaptive slow-fast system that dynamically activates and fuses LLM contributions based on scene complexity, achieving state-of-the-art performance in driving accuracy and computational efficiency on benchmarks.
Effectively integrating Large Language Models (LLMs) into autonomous driving requires a balance between leveraging high-level reasoning and maintaining real-time efficiency. Existing approaches either activate LLMs too frequently, causing excessive computational overhead, or use fixed schedules, failing to adapt to dynamic driving conditions. To address these challenges, we propose AdaDrive, an adaptively collaborative slow-fast framework that optimally determines when and how LLMs contribute to decision-making. (1) When to activate the LLM: AdaDrive employs a novel adaptive activation loss that dynamically determines LLM invocation based on a comparative learning mechanism, ensuring activation only in complex or critical scenarios. (2) How to integrate LLM assistance: Instead of rigid binary activation, AdaDrive introduces an adaptive fusion strategy that modulates a continuous, scaled LLM influence based on scene complexity and prediction confidence, ensuring seamless collaboration with conventional planners. Through these strategies, AdaDrive provides a flexible, context-aware framework that maximizes decision accuracy without compromising real-time performance. Extensive experiments on language-grounded autonomous driving benchmarks demonstrate that AdaDrive state-of-the-art performance in terms of both driving accuracy and computational efficiency. Code is available at https://github.com/ReaFly/AdaDrive.