Exploiting Hybrid Policy in Reinforcement Learning for Interpretable Temporal Logic Manipulation
This addresses efficiency and interpretability issues in robot learning for manipulation tasks, representing an incremental improvement over existing RL methods.
The paper tackles the low sampling efficiency and lack of semantic information in RL for long-horizon robot manipulation tasks by proposing a Temporal-Logic-guided Hybrid policy framework (HyTL), which improves performance and interpretability as demonstrated on four challenging tasks.
Reinforcement Learning (RL) based methods have been increasingly explored for robot learning. However, RL based methods often suffer from low sampling efficiency in the exploration phase, especially for long-horizon manipulation tasks, and generally neglect the semantic information from the task level, resulted in a delayed convergence or even tasks failure. To tackle these challenges, we propose a Temporal-Logic-guided Hybrid policy framework (HyTL) which leverages three-level decision layers to improve the agent's performance. Specifically, the task specifications are encoded via linear temporal logic (LTL) to improve performance and offer interpretability. And a waypoints planning module is designed with the feedback from the LTL-encoded task level as a high-level policy to improve the exploration efficiency. The middle-level policy selects which behavior primitives to execute, and the low-level policy specifies the corresponding parameters to interact with the environment. We evaluate HyTL on four challenging manipulation tasks, which demonstrate its effectiveness and interpretability. Our project is available at: https://sites.google.com/view/hytl-0257/.