Planning with Abstract Learned Models While Learning Transferable Subtasks
This work addresses the challenge of learning transferable hierarchical models in reinforcement learning, though it appears incremental as it builds on existing abstraction methods.
The authors tackled the problem of model-based hierarchical reinforcement learning by introducing PALM, which learns modular abstract models using L-AMDPs, resulting in efficient learning and improved transferability to new tasks.
We introduce an algorithm for model-based hierarchical reinforcement learning to acquire self-contained transition and reward models suitable for probabilistic planning at multiple levels of abstraction. We call this framework Planning with Abstract Learned Models (PALM). By representing subtasks symbolically using a new formal structure, the lifted abstract Markov decision process (L-AMDP), PALM learns models that are independent and modular. Through our experiments, we show how PALM integrates planning and execution, facilitating a rapid and efficient learning of abstract, hierarchical models. We also demonstrate the increased potential for learned models to be transferred to new and related tasks.