Prompt to Transfer: Sim-to-Real Transfer for Traffic Signal Control with Prompt Learning
This addresses the sim-to-real transfer problem for traffic signal control, which is incremental as it applies LLMs to a known bottleneck in RL-based transportation systems.
The paper tackles the performance gap when simulator-trained reinforcement learning policies for traffic signal control are deployed in the real world by using large language models to understand and profile system dynamics through prompt-based grounded action transformation, resulting in improved sim-to-real transfer as demonstrated with DQN experiments.
Numerous solutions are proposed for the Traffic Signal Control (TSC) tasks aiming to provide efficient transportation and mitigate congestion waste. In recent, promising results have been attained by Reinforcement Learning (RL) methods through trial and error in simulators, bringing confidence in solving cities' congestion headaches. However, there still exist performance gaps when simulator-trained policies are deployed to the real world. This issue is mainly introduced by the system dynamic difference between the training simulator and the real-world environments. The Large Language Models (LLMs) are trained on mass knowledge and proved to be equipped with astonishing inference abilities. In this work, we leverage LLMs to understand and profile the system dynamics by a prompt-based grounded action transformation. Accepting the cloze prompt template, and then filling in the answer based on accessible context, the pre-trained LLM's inference ability is exploited and applied to understand how weather conditions, traffic states, and road types influence traffic dynamics, being aware of this, the policies' action is taken and grounded based on realistic dynamics, thus help the agent learn a more realistic policy. We conduct experiments using DQN to show the effectiveness of the proposed PromptGAT's ability in mitigating the performance gap from simulation to reality (sim-to-real).