A Goal Without a Plan Is Just a Wish: Efficient and Effective Global Planner Training for Long-Horizon Agent Tasks
This addresses the challenge of improving planning for long-horizon agent tasks without human effort, offering an efficient solution, though it appears incremental as it builds on existing plan-and-execute frameworks.
The paper tackled the problem of LLM-based agents struggling with inefficient trial-and-error and hallucinatory actions in long-horizon tasks by introducing EAGLET, a plan-and-execute framework that trains a global planner, resulting in new state-of-the-art performance on three tasks and an 8x reduction in training costs compared to RL baselines.
Agents based on large language models (LLMs) struggle with brainless trial-and-error and generating hallucinatory actions due to a lack of global planning in long-horizon tasks. In this paper, we introduce a plan-and-execute framework and propose EAGLET, an efficient and effective planner training method to enhance the executor agent's planning abilities without human effort. Specifically, we train a plug-and-play global planner through a two-step process: we first synthesize high-quality plans from an advanced LLM using our proposed homologous consensus filtering strategy, and apply fine-tuning as a cold start. Moreover, we further improve the planner with a rule-based reinforcement learning stage using a novel executor capability gain reward, ensuring it can handle task instructions of varying difficulty. Experiments on three long-horizon agent tasks show that executor agents equipped with our planner outperform existing methods, achieving new state-of-the-art performance. Meanwhile, EAGLET reduces training costs by 8x compared to RL-based baselines, and it does not require manual effort or extra training data, offering an efficient and effective solution.