PABU: Progress-Aware Belief Update for Efficient LLM Agents
This addresses efficiency issues for LLM agents in tasks like those in AgentGym, though it is an incremental improvement over existing methods.
The paper tackled the problem of LLM agents using full action-observation histories, which leads to redundant actions and high inference costs, by proposing PABU, a belief-state framework that models task progress and selectively retains information, achieving an 81.0% task completion rate and reducing interaction steps by 26.9%.
Large Language Model (LLM) agents commonly condition actions on full action-observation histories, which introduce task-irrelevant information that easily leads to redundant actions and higher inference cost. We propose Progress-Aware Belief Update (PABU), a belief-state framework that compactly represents an agent's state by explicitly modeling task progress and selectively retaining past actions and observations. At each step, the agent predicts its relative progress since the previous round and decides whether the newly encountered interaction should be stored, conditioning future decisions only on the retained subset. Across eight environments in the AgentGym benchmark, and using identical training trajectories, PABU achieves an 81.0% task completion rate, outperforming previous State of the art (SoTA) models with full-history belief by 23.9%. Additionally, PABU's progress-oriented action selection improves efficiency, reducing the average number of interaction steps to 9.5, corresponding to a 26.9% reduction. Ablation studies show that both explicit progress prediction and selective retention are necessary for robust belief learning and performance gains.