Online Constrained Model-based Reinforcement Learning
This work addresses safe and efficient reinforcement learning for robotics, but it is incremental as it extends prior methods with online updates.
The authors tackled the problem of applying reinforcement learning to robotic systems with continuous state and action spaces, hard constraints, and limited resources by proposing a model-based approach combining Gaussian Process regression and Receding Horizon Control, resulting in an agent that learns and plans in real-time under non-linear constraints, as demonstrated on a cart pole swing-up and autonomous racing task with reusable dynamics from limited data.
Applying reinforcement learning to robotic systems poses a number of challenging problems. A key requirement is the ability to handle continuous state and action spaces while remaining within a limited time and resource budget. Additionally, for safe operation, the system must make robust decisions under hard constraints. To address these challenges, we propose a model based approach that combines Gaussian Process regression and Receding Horizon Control. Using sparse spectrum Gaussian Processes, we extend previous work by updating the dynamics model incrementally from a stream of sensory data. This results in an agent that can learn and plan in real-time under non-linear constraints. We test our approach on a cart pole swing-up environment and demonstrate the benefits of online learning on an autonomous racing task. The environment's dynamics are learned from limited training data and can be reused in new task instances without retraining.