Language Guided Exploration for RL Agents in Text Environments
This addresses the challenge of efficient learning for RL agents in complex text-based environments, representing an incremental improvement by integrating LLMs into RL exploration.
The paper tackles the problem of sparse rewards and large decision spaces in sequential decision making by introducing the Language Guided Exploration (LGE) framework, which uses a pre-trained language model to guide an RL agent, resulting in significant outperformance over vanilla RL and other methods on the ScienceWorld text environment.
Real-world sequential decision making is characterized by sparse rewards and large decision spaces, posing significant difficulty for experiential learning systems like $\textit{tabula rasa}$ reinforcement learning (RL) agents. Large Language Models (LLMs), with a wealth of world knowledge, can help RL agents learn quickly and adapt to distribution shifts. In this work, we introduce Language Guided Exploration (LGE) framework, which uses a pre-trained language model (called GUIDE ) to provide decision-level guidance to an RL agent (called EXPLORER). We observe that on ScienceWorld (Wang et al.,2022), a challenging text environment, LGE outperforms vanilla RL agents significantly and also outperforms other sophisticated methods like Behaviour Cloning and Text Decision Transformer.