GPUDrive: Data-driven, multi-agent driving simulation at 1 million FPS
This addresses the problem of scaling multi-agent learning for real-world planning applications, though it is incremental as it builds on existing simulation and GPU acceleration techniques.
The authors tackled the bottleneck of requiring billions of simulation steps for multi-agent planning by developing GPUDrive, a GPU-accelerated simulator that achieves over a million steps per second, enabling efficient training of reinforcement learning agents on the Waymo dataset in minutes to hours.
Multi-agent learning algorithms have been successful at generating superhuman planning in various games but have had limited impact on the design of deployed multi-agent planners. A key bottleneck in applying these techniques to multi-agent planning is that they require billions of steps of experience. To enable the study of multi-agent planning at scale, we present GPUDrive. GPUDrive is a GPU-accelerated, multi-agent simulator built on top of the Madrona Game Engine capable of generating over a million simulation steps per second. Observation, reward, and dynamics functions are written directly in C++, allowing users to define complex, heterogeneous agent behaviors that are lowered to high-performance CUDA. Despite these low-level optimizations, GPUDrive is fully accessible through Python, offering a seamless and efficient workflow for multi-agent, closed-loop simulation. Using GPUDrive, we train reinforcement learning agents on the Waymo Open Motion Dataset, achieving efficient goal-reaching in minutes and scaling to thousands of scenarios in hours. We open-source the code and pre-trained agents at https://github.com/Emerge-Lab/gpudrive.