A Multi-Agent, Policy-Gradient approach to Network Routing
This work addresses distributed decision-making in network routing, offering incremental improvements in convergence and cooperative learning.
The paper tackled network routing by applying a policy-gradient reinforcement learning algorithm (OLPOMDP) to simulated networks, achieving cooperative behavior among distributed agents without explicit communication and improving convergence rates through reward shaping.
Network routing is a distributed decision problem which naturally admits numerical performance measures, such as the average time for a packet to travel from source to destination. OLPOMDP, a policy-gradient reinforcement learning algorithm, was successfully applied to simulated network routing under a number of network models. Multiple distributed agents (routers) learned co-operative behavior without explicit inter-agent communication, and they avoided behavior which was individually desirable, but detrimental to the group's overall performance. Furthermore, shaping the reward signal by explicitly penalizing certain patterns of sub-optimal behavior was found to dramatically improve the convergence rate.