Zero-Shot Autonomous Vehicle Policy Transfer: From Simulation to Real-World via Adversarial Learning
For autonomous vehicle researchers, this work demonstrates a practical method for sim-to-real transfer that improves policy robustness without requiring real-world fine-tuning.
This paper achieves zero-shot transfer of an autonomous driving policy from simulation to a real-world scaled smart city using adversarial multi-agent reinforcement learning, outperforming human-driving baselines and adversary-free policies in travel time while improving robustness over Gaussian noise injection.
In this article, we demonstrate a zero-shot transfer of an autonomous driving policy from simulation to University of Delaware's scaled smart city with adversarial multi-agent reinforcement learning, in which an adversary attempts to decrease the net reward by perturbing both the inputs and outputs of the autonomous vehicles during training. We train the autonomous vehicles to coordinate with each other while crossing a roundabout in the presence of an adversary in simulation. The adversarial policy successfully reproduces the simulated behavior and incidentally outperforms, in terms of travel time, both a human-driving baseline and adversary-free trained policies. Finally, we demonstrate that the addition of adversarial training considerably improves the performance \eat{stability and robustness} of the policies after transfer to the real world compared to Gaussian noise injection.