Language Model as Planner and Formalizer under Constraints
This addresses safety concerns in downstream tasks by revealing limitations in LLM planning under realistic constraints, though it is incremental as it builds on existing benchmarks and methods.
The paper tackles the problem of overestimating LLM planning abilities due to simplistic benchmarks by augmenting them with fine-grained natural language constraints, showing that constraints consistently halve performance and challenge robustness across models and datasets.
LLMs have been widely used in planning, either as planners to generate action sequences end-to-end, or as formalizers to represent the planning domain and problem in a formal language that can derive plans deterministically. However, both lines of work rely on standard benchmarks that only include generic and simplistic environmental specifications, leading to potential overestimation of the planning ability of LLMs and safety concerns in downstream tasks. We bridge this gap by augmenting widely used planning benchmarks with manually annotated, fine-grained, and rich natural language constraints spanning four formally defined categories. Over 4 state-of-the-art reasoning LLMs, 3 formal languages, 5 methods, and 4 datasets, we show that the introduction of constraints not only consistently halves performance, but also significantly challenges robustness to problem complexity and lexical shift.