Learning to Drive via Asymmetric Self-Play
This addresses the challenge of expensive and unsafe data collection for autonomous driving by providing a scalable method to generate realistic training scenarios, though it is incremental as it builds on self-play and adversarial techniques.
The paper tackles the problem of scaling driving policy training beyond limited real-world data by proposing asymmetric self-play, which generates challenging synthetic scenarios to improve learning. The result is a policy with significantly fewer collisions in both nominal and long-tail scenarios, outperforming state-of-the-art adversarial approaches and real data alone in zero-shot transfer to end-to-end autonomy.
Large-scale data is crucial for learning realistic and capable driving policies. However, it can be impractical to rely on scaling datasets with real data alone. The majority of driving data is uninteresting, and deliberately collecting new long-tail scenarios is expensive and unsafe. We propose asymmetric self-play to scale beyond real data with additional challenging, solvable, and realistic synthetic scenarios. Our approach pairs a teacher that learns to generate scenarios it can solve but the student cannot, with a student that learns to solve them. When applied to traffic simulation, we learn realistic policies with significantly fewer collisions in both nominal and long-tail scenarios. Our policies further zero-shot transfer to generate training data for end-to-end autonomy, significantly outperforming state-of-the-art adversarial approaches, or using real data alone. For more information, visit https://waabi.ai/selfplay .