Foundation Models for Autonomous Driving System: An Initial Roadmap
This work addresses software engineering challenges for safety-critical autonomous driving systems, but it is incremental as it synthesizes existing knowledge into a roadmap without proposing new methods.
The paper tackles the integration of foundation models into autonomous driving systems by presenting an initial roadmap based on a literature review, identifying challenges and research opportunities across FM infrastructure, in-vehicle integration, and deployment.
Recent advances in foundation models (FMs), including large language models (LLMs), vision-language models (VLMs), and world models, have opened new opportunities for autonomous driving systems (ADSs) in perception, reasoning, decision-making, and interaction. However, ADSs are safety-critical cyber-physical systems, and integrating FMs into them raises substantial software engineering challenges in data curation, system design, deployment, evaluation, and assurance. To clarify this rapidly evolving landscape, we present an initial roadmap, grounded in a structured literature review, for integrating FMs into autonomous driving across three dimensions: FM infrastructure, in-vehicle integration, and practical deployment. For each dimension, we summarize the state of the art, identify key challenges, and highlight open research opportunities. Based on this analysis, we outline research directions for building reliable, safe, and trustworthy FM-enabled ADSs.