Safe Reinforcement Learning with Contrastive Risk Prediction
This addresses safety concerns in robotic applications of RL, but it is incremental as it builds on existing safe RL methods.
The paper tackles the problem of safety violations in reinforcement learning for robotics by proposing a risk preventive training method that uses a contrastive classifier to predict unsafe state-action probabilities, resulting in performance comparable to state-of-the-art model-based methods and outperforming conventional model-free approaches.
As safety violations can lead to severe consequences in real-world robotic applications, the increasing deployment of Reinforcement Learning (RL) in robotic domains has propelled the study of safe exploration for reinforcement learning (safe RL). In this work, we propose a risk preventive training method for safe RL, which learns a statistical contrastive classifier to predict the probability of a state-action pair leading to unsafe states. Based on the predicted risk probabilities, we can collect risk preventive trajectories and reshape the reward function with risk penalties to induce safe RL policies. We conduct experiments in robotic simulation environments. The results show the proposed approach has comparable performance with the state-of-the-art model-based methods and outperforms conventional model-free safe RL approaches.