From History to State: Constant-Context Skill Learning for LLM Agents
This work addresses the privacy-cost-capability tension in personal LLM agents by enabling efficient, privacy-preserving skill learning that reduces context length without sacrificing performance.
The paper proposes constant-context skill learning, a framework that moves recurring agent workflows from prompts into lightweight task-family modules, enabling LLM agents to achieve strong performance on ALFWorld, WebShop, and SciWorld while reducing prompt tokens per turn by 2–7×. With Qwen3-8B, SFT+RL achieves 89.6% unseen success on ALFWorld, 76.8% on WebShop, and 66.4% on SciWorld.
Large language model (LLM) agents are increasingly used to operate browsers, files, code and tools, making personal assistants a natural deployment target. Yet personal agents face a privacy-cost-capability tension: cloud models execute multi-step workflows well but expose sensitive intermediate context to external APIs, while local models preserve privacy but remain less reliable. Both settings also pay repeatedly for long skill prompts and growing histories. We propose constant-context skill learning, a context-to-weights framework for recurring agent workflows: reusable procedures are learned in lightweight task-family modules, while inference conditions only on the current observation and a compact state block. A deterministic tracker renders this state block from task progress and supplies aligned subgoal rewards, so each module can be trained with step-level SFT and refined through online RL. Across ALFWorld, WebShop, and SciWorld, our agents achieve strong performance across Qwen3-4B, Qwen3-8B and Llama-3.1-8B. With Qwen3-8B, SFT+RL reaches 89.6\% unseen success on ALFWorld, 76.8\% success on WebShop, and 66.4\% unseen success on SciWorld. They match or exceed strong published agent-training results while reducing prompt tokens per turn by 2--7$\times$ relative to controlled ReAct prompting baselines, showing that procedural context can be moved from prompts into weights.