Learning to Follow Instructions in Text-Based Games
This addresses the challenge of instruction following in text-based games for AI agents, representing an incremental improvement with a novel method for a known bottleneck.
The paper tackled the problem of reinforcement learning agents failing to follow natural language instructions in text-based games, showing that state-of-the-art agents performed poorly regardless of instructions. By using Linear Temporal Logic (LTL) to represent instructions temporally, their approach achieved superior performance in experiments with over 500 games in TextWorld.
Text-based games present a unique class of sequential decision making problem in which agents interact with a partially observable, simulated environment via actions and observations conveyed through natural language. Such observations typically include instructions that, in a reinforcement learning (RL) setting, can directly or indirectly guide a player towards completing reward-worthy tasks. In this work, we study the ability of RL agents to follow such instructions. We conduct experiments that show that the performance of state-of-the-art text-based game agents is largely unaffected by the presence or absence of such instructions, and that these agents are typically unable to execute tasks to completion. To further study and address the task of instruction following, we equip RL agents with an internal structured representation of natural language instructions in the form of Linear Temporal Logic (LTL), a formal language that is increasingly used for temporally extended reward specification in RL. Our framework both supports and highlights the benefit of understanding the temporal semantics of instructions and in measuring progress towards achievement of such a temporally extended behaviour. Experiments with 500+ games in TextWorld demonstrate the superior performance of our approach.