PokéAI: A Goal-Generating, Battle-Optimizing Multi-agent System for Pokemon Red
This is an incremental advancement in applying LLMs to game-playing, specifically for Pokémon Red, demonstrating potential for autonomous agents in text-based environments.
The authors tackled the problem of autonomously playing Pokémon Red by developing PokéAI, a multi-agent LLM framework with specialized agents for planning, execution, and critique, resulting in a battle module that achieved an 80.8% win rate in wild encounters, only 6% lower than an experienced human player.
We introduce PokéAI, the first text-based, multi-agent large language model (LLM) framework designed to autonomously play and progress through Pokémon Red. Our system consists of three specialized agents-Planning, Execution, and Critique-each with its own memory bank, role, and skill set. The Planning Agent functions as the central brain, generating tasks to progress through the game. These tasks are then delegated to the Execution Agent, which carries them out within the game environment. Upon task completion, the Critique Agent evaluates the outcome to determine whether the objective was successfully achieved. Once verification is complete, control returns to the Planning Agent, forming a closed-loop decision-making system. As a preliminary step, we developed a battle module within the Execution Agent. Our results show that the battle AI achieves an average win rate of 80.8% across 50 wild encounters, only 6% lower than the performance of an experienced human player. Furthermore, we find that a model's battle performance correlates strongly with its LLM Arena score on language-related tasks, indicating a meaningful link between linguistic ability and strategic reasoning. Finally, our analysis of gameplay logs reveals that each LLM exhibits a unique playstyle, suggesting that individual models develop distinct strategic behaviors.