Feasibility Consistent Representation Learning for Safe Reinforcement Learning
This work addresses safety constraint estimation in RL, which is crucial for real-world applications like robotics, but it appears incremental as it builds on existing representation learning methods.
The paper tackled the challenge of balancing safety constraints and reward performance in safe reinforcement learning by introducing the Feasibility Consistent Safe Reinforcement Learning (FCSRL) framework, which improved safety-aware embeddings and outperformed previous baselines in empirical evaluations.
In the field of safe reinforcement learning (RL), finding a balance between satisfying safety constraints and optimizing reward performance presents a significant challenge. A key obstacle in this endeavor is the estimation of safety constraints, which is typically more difficult than estimating a reward metric due to the sparse nature of the constraint signals. To address this issue, we introduce a novel framework named Feasibility Consistent Safe Reinforcement Learning (FCSRL). This framework combines representation learning with feasibility-oriented objectives to identify and extract safety-related information from the raw state for safe RL. Leveraging self-supervised learning techniques and a more learnable safety metric, our approach enhances the policy learning and constraint estimation. Empirical evaluations across a range of vector-state and image-based tasks demonstrate that our method is capable of learning a better safety-aware embedding and achieving superior performance than previous representation learning baselines.