Applications of Large Scale Foundation Models for Autonomous Driving
This work explores a novel application of foundation models to potentially improve autonomous driving, but it appears to be an incremental survey or conceptual framework rather than presenting new experimental results.
This paper investigates how large language models (LLMs) and foundation models can be applied to reformulate autonomous driving systems to address long-tailed AI challenges, categorizing techniques into simulation, world modeling, data annotation, and planning or end-to-end solutions.
Since DARPA Grand Challenges (rural) in 2004/05 and Urban Challenges in 2007, autonomous driving has been the most active field of AI applications. Recently powered by large language models (LLMs), chat systems, such as chatGPT and PaLM, emerge and rapidly become a promising direction to achieve artificial general intelligence (AGI) in natural language processing (NLP). There comes a natural thinking that we could employ these abilities to reformulate autonomous driving. By combining LLM with foundation models, it is possible to utilize the human knowledge, commonsense and reasoning to rebuild autonomous driving systems from the current long-tailed AI dilemma. In this paper, we investigate the techniques of foundation models and LLMs applied for autonomous driving, categorized as simulation, world model, data annotation and planning or E2E solutions etc.