Federated Hierarchical Reinforcement Learning for Adaptive Traffic Signal Control
This addresses inefficiencies in multi-agent reinforcement learning for traffic management by reducing communication needs and handling heterogeneity, though it is incremental as it builds on federated learning with clustering techniques.
The paper tackles the problem of adaptive traffic signal control in large-scale settings by proposing Hierarchical Federated Reinforcement Learning (HFRL), which dynamically groups intersections to improve coordination and scalability, outperforming decentralized and standard federated RL approaches in experiments on synthetic and real-world networks.
Multi-agent reinforcement learning (MARL) has shown promise for adaptive traffic signal control (ATSC), enabling multiple intersections to coordinate signal timings in real time. However, in large-scale settings, MARL faces constraints due to extensive data sharing and communication requirements. Federated learning (FL) mitigates these challenges by training shared models without directly exchanging raw data, yet traditional FL methods such as FedAvg struggle with highly heterogeneous intersections. Different intersections exhibit varying traffic patterns, demands, and road structures, so performing FedAvg across all agents is inefficient. To address this gap, we propose Hierarchical Federated Reinforcement Learning (HFRL) for ATSC. HFRL employs clustering-based or optimization-based techniques to dynamically group intersections and perform FedAvg independently within groups of intersections with similar characteristics, enabling more effective coordination and scalability than standard FedAvg. Our experiments on synthetic and real-world traffic networks demonstrate that HFRL not only outperforms both decentralized and standard federated RL approaches but also identifies suitable grouping patterns based on network structure or traffic demand, resulting in a more robust framework for distributed, heterogeneous systems.