LeAD: The LLM Enhanced Planning System Converged with End-to-end Autonomous Driving
This addresses the challenge of handling edge cases in autonomous driving for urban deployment, though it appears incremental as it combines existing methods.
The paper tackles the problem of autonomous driving systems failing in complex urban scenarios by integrating an end-to-end framework with a large language model (LLM) for enhanced reasoning, achieving 71 points on the CARLA Leaderboard V1 benchmark with 93% route completion.
A principal barrier to large-scale deployment of urban autonomous driving systems lies in the prevalence of complex scenarios and edge cases. Existing systems fail to effectively interpret semantic information within traffic contexts and discern intentions of other participants, consequently generating decisions misaligned with skilled drivers' reasoning patterns. We present LeAD, a dual-rate autonomous driving architecture integrating imitation learning-based end-to-end (E2E) frameworks with large language model (LLM) augmentation. The high-frequency E2E subsystem maintains real-time perception-planning-control cycles, while the low-frequency LLM module enhances scenario comprehension through multi-modal perception fusion with HD maps and derives optimal decisions via chain-of-thought (CoT) reasoning when baseline planners encounter capability limitations. Our experimental evaluation in the CARLA Simulator demonstrates LeAD's superior handling of unconventional scenarios, achieving 71 points on Leaderboard V1 benchmark, with a route completion of 93%.