Using a Deep Reinforcement Learning Agent for Traffic Signal Control
This work addresses traffic efficiency for urban transportation systems, representing an incremental improvement with specific gains in simulation.
The authors tackled traffic signal control by developing a deep reinforcement learning agent with a novel discrete traffic state encoding, achieving reductions of 82% in average cumulative delay, 66% in average queue length, and 20% in average travel time compared to a baseline neural network agent.
Ensuring transportation systems are efficient is a priority for modern society. Technological advances have made it possible for transportation systems to collect large volumes of varied data on an unprecedented scale. We propose a traffic signal control system which takes advantage of this new, high quality data, with minimal abstraction compared to other proposed systems. We apply modern deep reinforcement learning methods to build a truly adaptive traffic signal control agent in the traffic microsimulator SUMO. We propose a new state space, the discrete traffic state encoding, which is information dense. The discrete traffic state encoding is used as input to a deep convolutional neural network, trained using Q-learning with experience replay. Our agent was compared against a one hidden layer neural network traffic signal control agent and reduces average cumulative delay by 82%, average queue length by 66% and average travel time by 20%.