Centralized Cooperation for Connected and Automated Vehicles at Intersections by Proximal Policy Optimization
This addresses computational bottlenecks for real-time traffic management in autonomous vehicle systems, though it is incremental as it builds on existing RL methods.
The paper tackled the problem of low computational efficiency in centralized coordination of automated vehicles at intersections by proposing a reinforcement learning approach, which reduced computing time to 1/400 of a model predictive control method and increased intersection efficiency by 4.5 times.
Connected vehicles will change the modes of future transportation management and organization, especially at an intersection without traffic light. Centralized coordination methods globally coordinate vehicles approaching the intersection from all sections by considering their states altogether. However, they need substantial computation resources since they own a centralized controller to optimize the trajectories for all approaching vehicles in real-time. In this paper, we propose a centralized coordination scheme of automated vehicles at an intersection without traffic signals using reinforcement learning (RL) to address low computation efficiency suffered by current centralized coordination methods. We first propose an RL training algorithm, model accelerated proximal policy optimization (MA-PPO), which incorporates a prior model into proximal policy optimization (PPO) algorithm to accelerate the learning process in terms of sample efficiency. Then we present the design of state, action and reward to formulate centralized coordination as an RL problem. Finally, we train a coordinate policy in a simulation setting and compare computing time and traffic efficiency with a coordination scheme based on model predictive control (MPC) method. Results show that our method spends only 1/400 of the computing time of MPC and increase the efficiency of the intersection by 4.5 times.