Shielded Reinforcement Learning Under Dynamic Temporal Logic Constraints
This work addresses the problem of deploying RL on real systems with complex operational constraints for robotics applications, representing an incremental advance in safe RL by extending beyond traditional safety to rich STL specifications.
The paper tackles the challenge of enforcing complex spatio-temporal tasks, such as periodic recharging or time-bounded visits, during reinforcement learning (RL) by proposing a framework that uses sequential control barrier functions and model-free RL to ensure satisfaction of Signal Temporal Logic (STL) specifications, including dynamic targets with unknown trajectories, and demonstrates its effectiveness through simulations.
Reinforcement Learning (RL) has shown promise in various robotics applications, yet its deployment on real systems is still limited due to safety and operational constraints. The safe RL field has gained considerable attention in recent years, which focuses on imposing safety constraints throughout the learning process. However, real systems often require more complex constraints than just safety, such as periodic recharging or time-bounded visits to specific regions. Imposing such spatio-temporal tasks during learning still remains a challenge. Signal Temporal Logic (STL) is a formal language for specifying temporal properties of real-valued signals and provides a way to express such complex tasks. In this paper, we propose a framework that leverages sequential control barrier functions and model-free RL to ensure that the given STL tasks are satisfied throughout the learning process. Our method extends beyond traditional safety constraints by enforcing rich STL specifications, which can involve visits to dynamic targets with unknown trajectories. We also demonstrate the effectiveness of our framework through various simulations.