Drive Like a Human: Rethinking Autonomous Driving with Large Language Models
This addresses the problem of improving autonomous driving safety and adaptability in complex scenarios for developers and users, though it is incremental as it builds on existing LLM capabilities.
The paper tackles the limitations of traditional autonomous driving systems in handling long-tail corner cases by proposing a human-like approach using a large language model (LLM) for reasoning, interpretation, and memorization, demonstrating its feasibility through a closed-loop system and experiments showing it can solve such cases.
In this paper, we explore the potential of using a large language model (LLM) to understand the driving environment in a human-like manner and analyze its ability to reason, interpret, and memorize when facing complex scenarios. We argue that traditional optimization-based and modular autonomous driving (AD) systems face inherent performance limitations when dealing with long-tail corner cases. To address this problem, we propose that an ideal AD system should drive like a human, accumulating experience through continuous driving and using common sense to solve problems. To achieve this goal, we identify three key abilities necessary for an AD system: reasoning, interpretation, and memorization. We demonstrate the feasibility of employing an LLM in driving scenarios by building a closed-loop system to showcase its comprehension and environment-interaction abilities. Our extensive experiments show that the LLM exhibits the impressive ability to reason and solve long-tailed cases, providing valuable insights for the development of human-like autonomous driving. The related code are available at https://github.com/PJLab-ADG/DriveLikeAHuman .