Generator-Assistant Stepwise Rollback Framework for Large Language Model Agent
This addresses error accumulation in LLM agents, which is crucial for reliable task completion, though it is an incremental improvement on existing reasoning frameworks.
The paper tackles the one-pass error propagation problem in step-by-step reasoning for large language model agents by proposing the GA-Rollback framework, which uses a generator and assistant to detect and rollback incorrect actions, achieving significant improvements over baselines on three benchmarks.
Large language model (LLM) agents typically adopt a step-by-step reasoning framework, in which they interleave the processes of thinking and acting to accomplish the given task. However, this paradigm faces a deep-rooted one-pass issue whereby each generated intermediate thought is plugged into the trajectory regardless of its correctness, which can cause irreversible error propagation. To address the issue, this paper proposes a novel framework called Generator-Assistant Stepwise Rollback (GA-Rollback) to induce better decision-making for LLM agents. Particularly, GA-Rollback utilizes a generator to interact with the environment and an assistant to examine each action produced by the generator, where the assistant triggers a rollback operation upon detection of incorrect actions. Moreover, we introduce two additional strategies tailored for the rollback scenario to further improve its effectiveness. Extensive experiments show that GA-Rollback achieves significant improvements over several strong baselines on three widely used benchmarks. Our analysis further reveals that GA-Rollback can function as a robust plug-and-play module, integrating seamlessly with other methods.