AMM: Adaptive Modularized Reinforcement Model for Multi-city Traffic Signal Control
This work addresses the problem of high experimental costs in real-world traffic signal control for urban planners and AI researchers, though it is incremental as it builds on existing domain adaptation and meta-learning approaches.
The paper tackles the challenge of applying reinforcement learning to multi-city traffic signal control by proposing an adaptive modularized model that addresses differences across cities and improves data utilization, achieving excellent performance with limited interactions in target environments and outperforming existing methods.
Traffic signal control (TSC) is an important and widely studied direction. Recently, reinforcement learning (RL) methods have been used to solve TSC problems and achieve superior performance over conventional TSC methods. However, applying RL methods to the real world is challenging due to the huge cost of experiments in real-world traffic environments. One possible solution is TSC domain adaptation, which adapts trained models to target environments and reduces the number of interactions and the training cost. However, existing TSC domain adaptation methods still face two major issues: the lack of consideration for differences across cities and the low utilization of multi-city data. To solve aforementioned issues, we propose an approach named Adaptive Modularized Model (AMM). By modularizing TSC problems and network models, we overcome the challenge of possible changes in environmental observations. We also aggregate multi-city experience through meta-learning. We conduct extensive experiments on different cities and show that AMM can achieve excellent performance with limited interactions in target environments and outperform existing methods. We also demonstrate the feasibility and generalizability of our method.