Lyapunov Function Consistent Adaptive Network Signal Control with Back Pressure and Reinforcement Learning
This addresses traffic congestion for urban networks by providing an adaptive signal control method, though it appears incremental as it builds on existing back-pressure and RL approaches.
The study tackled traffic signal control by unifying flow-based and pressure-based methods using Lyapunov control theory and improved them with traffic flow theory, then designed a reward function for RL-based control; numerical tests showed it outperformed alternative methods in reducing average network vehicle waiting time across various traffic scenarios.
In traffic signal control, flow-based (optimizing the overall flow) and pressure-based methods (equalizing and alleviating congestion) are commonly used but often considered separately. This study introduces a unified framework using Lyapunov control theory, defining specific Lyapunov functions respectively for these methods. We have found interesting results. For example, the well-recognized back-pressure method is equal to differential queue lengths weighted by intersection lane saturation flows. We further improve it by adding basic traffic flow theory. Rather than ensuring that the control system be stable, the system should be also capable of adaptive to various performance metrics. Building on insights from Lyapunov theory, this study designs a reward function for the Reinforcement Learning (RL)-based network signal control, whose agent is trained with Double Deep Q-Network (DDQN) for effective control over complex traffic networks. The proposed algorithm is compared with several traditional and RL-based methods under pure passenger car flow and heterogenous traffic flow including freight, respectively. The numerical tests demonstrate that the proposed method outperforms the alternative control methods across different traffic scenarios, covering corridor and general network situations each with varying traffic demands, in terms of the average network vehicle waiting time per vehicle.