PG3: Policy-Guided Planning for Generalized Policy Generation
This work addresses a longstanding challenge in classical planning for AI researchers, offering an incremental improvement over existing methods for generalized policy generation.
The paper tackles the problem of synthesizing policies that generalize across multiple planning problems by proposing PG3, a new policy-guided planning approach that evaluates candidate policies through guided planning on training problems. Empirical results in six domains show that PG3 learns generalized policies more efficiently and effectively than baselines, with specific gains in performance metrics.
A longstanding objective in classical planning is to synthesize policies that generalize across multiple problems from the same domain. In this work, we study generalized policy search-based methods with a focus on the score function used to guide the search over policies. We demonstrate limitations of two score functions and propose a new approach that overcomes these limitations. The main idea behind our approach, Policy-Guided Planning for Generalized Policy Generation (PG3), is that a candidate policy should be used to guide planning on training problems as a mechanism for evaluating that candidate. Theoretical results in a simplified setting give conditions under which PG3 is optimal or admissible. We then study a specific instantiation of policy search where planning problems are PDDL-based and policies are lifted decision lists. Empirical results in six domains confirm that PG3 learns generalized policies more efficiently and effectively than several baselines. Code: https://github.com/ryangpeixu/pg3