Monte Carlo Planning with Large Language Model for Text-Based Game Agents
This work addresses the problem of time-consuming planning for language-based agents in text-based games, offering a more efficient solution, though it is incremental as it builds on existing tree search and LLM techniques.
The paper tackles the inefficiency of planning-then-learning paradigms in text-based games by introducing MC-DML, which combines LLMs with Monte Carlo planning and dynamic memory, resulting in significant performance improvements across games in the Jericho benchmark, outperforming methods that require multiple iterations.
Text-based games provide valuable environments for language-based autonomous agents. However, planning-then-learning paradigms, such as those combining Monte Carlo Tree Search (MCTS) and reinforcement learning (RL), are notably time-consuming due to extensive iterations. Additionally, these algorithms perform uncertainty-driven exploration but lack language understanding and reasoning abilities. In this paper, we introduce the Monte Carlo planning with Dynamic Memory-guided Large language model (MC-DML) algorithm. MC-DML leverages the language understanding and reasoning capabilities of Large Language Models (LLMs) alongside the exploratory advantages of tree search algorithms. Specifically, we enhance LLMs with in-trial and cross-trial memory mechanisms, enabling them to learn from past experiences and dynamically adjust action evaluations during planning. We conduct experiments on a series of text-based games from the Jericho benchmark. Our results demonstrate that the MC-DML algorithm significantly enhances performance across various games at the initial planning phase, outperforming strong contemporary methods that require multiple iterations. This demonstrates the effectiveness of our algorithm, paving the way for more efficient language-grounded planning in complex environments.