AutoGuide: Automated Generation and Selection of Context-Aware Guidelines for Large Language Model Agents
This addresses the challenge of enhancing LLM agents' decision-making in domains where they lack knowledge, offering a novel approach beyond demonstration-based learning.
The paper tackles the problem of guiding large language models in unfamiliar domains like web navigation by introducing AutoGuide, a framework that automatically generates context-aware guidelines from offline experiences, resulting in significant performance improvements over baselines in complex benchmark domains.
Recent advances in large language models (LLMs) have empowered AI agents capable of performing various sequential decision-making tasks. However, effectively guiding LLMs to perform well in unfamiliar domains like web navigation, where they lack sufficient knowledge, has proven to be difficult with the demonstration-based in-context learning paradigm. In this paper, we introduce a novel framework, called AutoGuide, which addresses this limitation by automatically generating context-aware guidelines from offline experiences. Importantly, each context-aware guideline is expressed in concise natural language and follows a conditional structure, clearly describing the context where it is applicable. As a result, our guidelines facilitate the provision of relevant knowledge for the agent's current decision-making process, overcoming the limitations of the conventional demonstration-based learning paradigm. Our evaluation demonstrates that AutoGuide significantly outperforms competitive baselines in complex benchmark domains, including real-world web navigation.