Unifying Inference-Time Planning Language Generation
This work unifies inference-time planning language generation for classical planning, providing recipes and conclusions to improve trust and performance in AI systems.
The authors tackled the problem of scattered methods in LLM-based planning language generation by proposing a unifying framework based on intermediate representations, systematically evaluating over a dozen pipelines and showing their robustness against problem complexity.
A line of work in planning uses LLM not to generate a plan, but to generate a formal representation in some planning language, which can be input into a symbolic solver to deterministically find a plan. While showing improved trust and promising performance, dozens of recent publications have proposed scattered methods on a variety of benchmarks under different experimental settings. We attempt to unify the inference-time LLM-as-formalizer methodology for classical planning by proposing a unifying framework based on intermediate representations. We thus systematically evaluate more than a dozen pipelines that subsume most existing work, while proposing novel ones that involve syntactically similar but high resource intermediate languages (such as a Python wrapper of PDDL). We provide recipes for planning language generation pipelines, draw a series of conclusions showing the efficacy of their various components, and evidence their robustness against problem complexity.