Data Generation Method for Learning a Low-dimensional Safe Region in Safe Reinforcement Learning
This addresses data reliability issues in safe reinforcement learning for nonlinear dynamical systems, but it appears incremental as it builds on existing feature extraction methods.
The paper tackles the problem of generating training data for safe reinforcement learning in high-dimensional systems by proposing a method that combines two sampling approaches to balance learning performance and safety risk, demonstrating it on a three-link inverted pendulum example.
Safe reinforcement learning aims to learn a control policy while ensuring that neither the system nor the environment gets damaged during the learning process. For implementing safe reinforcement learning on highly nonlinear and high-dimensional dynamical systems, one possible approach is to find a low-dimensional safe region via data-driven feature extraction methods, which provides safety estimates to the learning algorithm. As the reliability of the learned safety estimates is data-dependent, we investigate in this work how different training data will affect the safe reinforcement learning approach. By balancing between the learning performance and the risk of being unsafe, a data generation method that combines two sampling methods is proposed to generate representative training data. The performance of the method is demonstrated with a three-link inverted pendulum example.