Robust Hierarchical Planning with Policy Delegation
This work addresses planning efficiency and adaptability for AI systems, representing an incremental improvement in hierarchical planning techniques.
The paper tackles the problem of hierarchical planning by introducing a delegation-based framework called the Markov Intent Process, which dynamically creates skill hierarchies and reduces computational overhead. The approach demonstrates competitive performance against classic planning and reinforcement learning methods in terms of solution length and planning time across various domains.
We propose a novel framework and algorithm for hierarchical planning based on the principle of delegation. This framework, the Markov Intent Process, features a collection of skills which are each specialised to perform a single task well. Skills are aware of their intended effects and are able to analyse planning goals to delegate planning to the best-suited skill. This principle dynamically creates a hierarchy of plans, in which each skill plans for sub-goals for which it is specialised. The proposed planning method features on-demand execution---skill policies are only evaluated when needed. Plans are only generated at the highest level, then expanded and optimised when the latest state information is available. The high-level plan retains the initial planning intent and previously computed skills, effectively reducing the computation needed to adapt to environmental changes. We show this planning approach is experimentally very competitive to classic planning and reinforcement learning techniques on a variety of domains, both in terms of solution length and planning time.