Constrained Meta Reinforcement Learning with Provable Test-Time Safety
This addresses safety-critical applications like robotics and healthcare by providing a framework for safe and efficient learning in meta RL, though it is incremental as it builds on existing constrained meta RL methods.
The paper tackles the problem of ensuring safety during testing in constrained meta reinforcement learning, proposing an algorithm that refines policies with provable safety and sample complexity guarantees for learning near-optimal policies on test tasks, and shows this sample complexity is tight with a matching lower bound.
Meta reinforcement learning (RL) allows agents to leverage experience across a distribution of tasks on which the agent can train at will, enabling faster learning of optimal policies on new test tasks. Despite its success in improving sample complexity on test tasks, many real-world applications, such as robotics and healthcare, impose safety constraints during testing. Constrained meta RL provides a promising framework for integrating safety into meta RL. An open question in constrained meta RL is how to ensure the safety of the policy on the real-world test task, while reducing the sample complexity and thus, enabling faster learning of optimal policies. To address this gap, we propose an algorithm that refines policies learned during training, with provable safety and sample complexity guarantees for learning a near optimal policy on the test tasks. We further derive a matching lower bound, showing that this sample complexity is tight.