Uncertainty-aware Grounded Action Transformation towards Sim-to-Real Transfer for Traffic Signal Control
This addresses the sim-to-real transfer problem for traffic signal control, which impacts urban traffic management, but it is incremental as it builds on existing RL methods.
The paper tackles the performance gap between simulation and real-world environments in reinforcement learning for traffic signal control by proposing UGAT, a method that dynamically transforms actions with uncertainty, resulting in significant improvement in transferred policy performance.
Traffic signal control (TSC) is a complex and important task that affects the daily lives of millions of people. Reinforcement Learning (RL) has shown promising results in optimizing traffic signal control, but current RL-based TSC methods are mainly trained in simulation and suffer from the performance gap between simulation and the real world. In this paper, we propose a simulation-to-real-world (sim-to-real) transfer approach called UGAT, which transfers a learned policy trained from a simulated environment to a real-world environment by dynamically transforming actions in the simulation with uncertainty to mitigate the domain gap of transition dynamics. We evaluate our method on a simulated traffic environment and show that it significantly improves the performance of the transferred RL policy in the real world.