How To Avoid Being Eaten By a Grue: Exploration Strategies for Text-Adventure Agents
This work addresses a specific bottleneck in text-adventure agents for AI researchers, but it is incremental as it builds on prior knowledge graph methods.
The paper tackles the problem of large combinatorial action spaces in text-based games, which cause poor sample efficiency and prevent agents from passing bottleneck states; the authors introduce two new exploration strategies that enable agents to pass the Grue bottleneck in Zork1, where prior agents failed.
Text-based games -- in which an agent interacts with the world through textual natural language -- present us with the problem of combinatorially-sized action-spaces. Most current reinforcement learning algorithms are not capable of effectively handling such a large number of possible actions per turn. Poor sample efficiency, consequently, results in agents that are unable to pass bottleneck states, where they are unable to proceed because they do not see the right action sequence to pass the bottleneck enough times to be sufficiently reinforced. Building on prior work using knowledge graphs in reinforcement learning, we introduce two new game state exploration strategies. We compare our exploration strategies against strong baselines on the classic text-adventure game, Zork1, where prior agent have been unable to get past a bottleneck where the agent is eaten by a Grue.