Learning Classical Planning Strategies with Policy Gradient
This work addresses the need for adaptive search strategies in classical planning, offering domain-specific improvements, though it is incremental as it builds on existing heuristic forward search methods.
The paper tackles the problem of fixed search strategies in classical planning by introducing a trainable framework that alternates between multiple forward search approaches using a policy gradient method, resulting in improved performance relative to baselines like best-first search and uniform policies across five IPC domains.
A common paradigm in classical planning is heuristic forward search. Forward search planners often rely on simple best-first search which remains fixed throughout the search process. In this paper, we introduce a novel search framework capable of alternating between several forward search approaches while solving a particular planning problem. Selection of the approach is performed using a trainable stochastic policy, mapping the state of the search to a probability distribution over the approaches. This enables using policy gradient to learn search strategies tailored to a specific distributions of planning problems and a selected performance metric, e.g. the IPC score. We instantiate the framework by constructing a policy space consisting of five search approaches and a two-dimensional representation of the planner's state. Then, we train the system on randomly generated problems from five IPC domains using three different performance metrics. Our experimental results show that the learner is able to discover domain-specific search strategies, improving the planner's performance relative to the baselines of plain best-first search and a uniform policy.