Planning in the Dark: LLM-Symbolic Planning Pipeline without Experts
This enables broader access to AI planning by eliminating the need for expert domain knowledge, though it is an incremental improvement over existing hybrid methods.
The paper tackles the problem of inconsistent reasoning and hallucination in LLM-based planning by proposing a fully automated pipeline that generates multiple action schema candidates and validates them without expert intervention, achieving superior planning performance compared to direct LLM approaches.
Large Language Models (LLMs) have shown promise in solving natural language-described planning tasks, but their direct use often leads to inconsistent reasoning and hallucination. While hybrid LLM-symbolic planning pipelines have emerged as a more robust alternative, they typically require extensive expert intervention to refine and validate generated action schemas. It not only limits scalability but also introduces a potential for biased interpretation, as a single expert's interpretation of ambiguous natural language descriptions might not align with the user's actual intent. To address this, we propose a novel approach that constructs an action schema library to generate multiple candidates, accounting for the diverse possible interpretations of natural language descriptions. We further introduce a semantic validation and ranking module that automatically filter and rank the generated schemas and plans without expert-in-the-loop. The experiments showed our pipeline maintains superiority in planning over the direct LLM planning approach. These findings demonstrate the feasibility of a fully automated end-to-end LLM-symbolic planner that requires no expert intervention, opening up the possibility for a broader audience to engage with AI planning with less prerequisite of domain expertise.