Multi-agent Reinforcement Learning for Networked System Control
It addresses control problems in networked systems like traffic and vehicle coordination, offering incremental improvements over prior communication protocols.
This paper tackles multi-agent reinforcement learning for networked system control by introducing a spatial discount factor and a differentiable communication protocol called NeurComm, which improve learning stability and outperform existing methods in traffic signal and cruise control scenarios with enhanced efficiency and performance.
This paper considers multi-agent reinforcement learning (MARL) in networked system control. Specifically, each agent learns a decentralized control policy based on local observations and messages from connected neighbors. We formulate such a networked MARL (NMARL) problem as a spatiotemporal Markov decision process and introduce a spatial discount factor to stabilize the training of each local agent. Further, we propose a new differentiable communication protocol, called NeurComm, to reduce information loss and non-stationarity in NMARL. Based on experiments in realistic NMARL scenarios of adaptive traffic signal control and cooperative adaptive cruise control, an appropriate spatial discount factor effectively enhances the learning curves of non-communicative MARL algorithms, while NeurComm outperforms existing communication protocols in both learning efficiency and control performance.