$How^{2}$: How to learn from procedural How-to questions
This addresses the problem of inefficient planning for AI agents in interactive environments, though it is incremental as it builds on existing memory and LLM-based agent frameworks.
The paper tackles the challenge of AI agents learning from procedural how-to questions by introducing the $How^{2}$ framework, which enables agents to ask, store, and reuse answers for lifelong planning, showing in a Minecraft environment that abstract answers improve learning efficiency.
An agent facing a planning problem can use answers to how-to questions to reduce uncertainty and fill knowledge gaps, helping it solve both current and future tasks. However, their open ended nature, where valid answers to "How do I X?" range from executable actions to high-level descriptions of X's sub-goals, makes them challenging for AI agents to ask, and for AI experts to answer, in ways that support efficient planning. We introduce $How^{2}$, a memory agent framework that enables agents to ask how-to questions, store the answers, and reuse them for lifelong learning in interactive environments. We evaluate our approach in Plancraft, a Minecraft crafting environment, where agents must complete an assembly task by manipulating inventory items. Using teacher models that answer at varying levels of abstraction, from executable action sequences to high-level subgoal descriptions, we show that lifelong learning agents benefit most from answers that are abstracted and decoupled from the current state. $How^{2}$ offers a way for LLM-based agents to improve their planning capabilities over time by asking questions in interactive environments.