Improving Agent Behaviors with RL Fine-tuning for Autonomous Driving
This work addresses reliability issues in autonomous vehicle simulations and planning, though it is incremental as it builds on existing supervised learning methods with RL fine-tuning.
The paper tackles the problem of distribution shift in supervised learning models for autonomous driving agent behaviors by using reinforcement learning for closed-loop fine-tuning, resulting in improved overall performance and reduced collision rates on the Waymo Open Sim Agents challenge.
A major challenge in autonomous vehicle research is modeling agent behaviors, which has critical applications including constructing realistic and reliable simulations for off-board evaluation and forecasting traffic agents motion for onboard planning. While supervised learning has shown success in modeling agents across various domains, these models can suffer from distribution shift when deployed at test-time. In this work, we improve the reliability of agent behaviors by closed-loop fine-tuning of behavior models with reinforcement learning. Our method demonstrates improved overall performance, as well as improved targeted metrics such as collision rate, on the Waymo Open Sim Agents challenge. Additionally, we present a novel policy evaluation benchmark to directly assess the ability of simulated agents to measure the quality of autonomous vehicle planners and demonstrate the effectiveness of our approach on this new benchmark.