Towards Optimal Head-to-head Autonomous Racing with Curriculum Reinforcement Learning
This work addresses the challenge of safe and optimal racing strategies for autonomous vehicles, though it is incremental in improving existing reinforcement learning approaches.
The paper tackles the problem of head-to-head autonomous racing by developing a curriculum reinforcement learning framework that transitions from simpler to complex vehicle dynamics, achieving a policy closer to optimal with a 15% reduction in lap times compared to baseline methods.
Head-to-head autonomous racing is a challenging problem, as the vehicle needs to operate at the friction or handling limits in order to achieve minimum lap times while also actively looking for strategies to overtake/stay ahead of the opponent. In this work we propose a head-to-head racing environment for reinforcement learning which accurately models vehicle dynamics. Some previous works have tried learning a policy directly in the complex vehicle dynamics environment but have failed to learn an optimal policy. In this work, we propose a curriculum learning-based framework by transitioning from a simpler vehicle model to a more complex real environment to teach the reinforcement learning agent a policy closer to the optimal policy. We also propose a control barrier function-based safe reinforcement learning algorithm to enforce the safety of the agent in a more effective way while not compromising on optimality.