What's the Plan? Evaluating and Developing Planning-Aware Techniques for Language Models
This addresses the planning limitations of LLMs for AI applications, presenting an incremental hybrid approach.
The paper tackles the problem that large language models (LLMs) lack necessary planning skills for applications like web or embodied agents, and introduces SimPlan, a hybrid method combining LLMs with classical planning, which significantly outperforms existing LLM-based planners in experiments across various domains.
Planning is a fundamental task in artificial intelligence that involves finding a sequence of actions that achieve a specified goal in a given environment. Large language models (LLMs) are increasingly used for applications that require planning capabilities, such as web or embodied agents. In line with recent studies, we demonstrate through experimentation that LLMs lack necessary skills required for planning. Based on these observations, we advocate for the potential of a hybrid approach that combines LLMs with classical planning methodology. Then, we introduce SimPlan, a novel hybrid-method, and evaluate its performance in a new challenging setup. Our extensive experiments across various planning domains demonstrate that SimPlan significantly outperforms existing LLM-based planners.