Empowering Autonomous Driving with Large Language Models: A Safety Perspective
This addresses safety issues in autonomous driving for real-world applications, though it is incremental as it builds on existing LLM and AD methods.
The paper tackles safety challenges in autonomous driving by integrating Large Language Models (LLMs) into decision-making, showing superior performance and safety metrics compared to state-of-the-art approaches in simulated environments.
Autonomous Driving (AD) encounters significant safety hurdles in long-tail unforeseen driving scenarios, largely stemming from the non-interpretability and poor generalization of the deep neural networks within the AD system, particularly in out-of-distribution and uncertain data. To this end, this paper explores the integration of Large Language Models (LLMs) into AD systems, leveraging their robust common-sense knowledge and reasoning abilities. The proposed methodologies employ LLMs as intelligent decision-makers in behavioral planning, augmented with a safety verifier shield for contextual safety learning, for enhancing driving performance and safety. We present two key studies in a simulated environment: an adaptive LLM-conditioned Model Predictive Control (MPC) and an LLM-enabled interactive behavior planning scheme with a state machine. Demonstrating superior performance and safety metrics compared to state-of-the-art approaches, our approach shows the promising potential for using LLMs for autonomous vehicles.