On Tackling Complex Tasks with Reward Machines and Signal Temporal Logics
This work addresses the challenge of specifying and learning complex temporal tasks in RL, offering a more structured representation, but the contribution is incremental as it extends existing RM and STL concepts.
The paper proposes a reinforcement learning framework that combines Reward Machines with Signal Temporal Logic to efficiently represent complex tasks and guide training toward satisfying specified requirements, demonstrated on three case studies.
We propose a Reinforcement Learning (RL) based control design framework for handling complex tasks. The approach extends the concept of Reward Machines (RM) with Signal Temporal Logic (STL) formulas that can be used for event generation. The use of STL allows not only a more efficient representation of rewards for complex tasks but also guiding the training process to converge towards behaviors satisfying specified requirements. We also propose an implementation of the framework that leverages the STL online monitoring algorithms. We illustrate the framework with three case studies (minigrid, cart-pole and high-way environments) with non-trivial tasks.