Model-based Constrained Reinforcement Learning using Generalized Control Barrier Function
This addresses safety and efficiency issues in autonomous driving by reducing constraint violations, though it is incremental as it builds on existing constrained RL approaches.
The paper tackles the problem of constraint violations in reinforcement learning for real-world tasks like autonomous driving by proposing a model-based feasibility enhancement technique using generalized control barrier functions. The result is up to four times fewer constraint violations and 3.36 times faster convergence compared to baseline methods.
Model information can be used to predict future trajectories, so it has huge potential to avoid dangerous region when implementing reinforcement learning (RL) on real-world tasks, like autonomous driving. However, existing studies mostly use model-free constrained RL, which causes inevitable constraint violations. This paper proposes a model-based feasibility enhancement technique of constrained RL, which enhances the feasibility of policy using generalized control barrier function (GCBF) defined on the distance to constraint boundary. By using the model information, the policy can be optimized safely without violating actual safety constraints, and the sample efficiency is increased. The major difficulty of infeasibility in solving the constrained policy gradient is handled by an adaptive coefficient mechanism. We evaluate the proposed method in both simulations and real vehicle experiments in a complex autonomous driving collision avoidance task. The proposed method achieves up to four times fewer constraint violations and converges 3.36 times faster than baseline constrained RL approaches.