A Mixture-of-Experts Approach to Few-Shot Task Transfer in Open-Ended Text Worlds
This addresses the challenge of rapid task adaptation for agents in open-ended environments, though it is incremental as it builds on existing mixture-of-experts and attention methods.
The paper tackled the problem of few-shot task transfer in open-ended text worlds by introducing a Mixture-of-Experts model with attention across frozen and unfrozen experts, resulting in higher rewards in zero-shot settings and greater sample efficiency in few-shot learning.
Open-ended worlds are those in which there are no pre-specified goals or environmental reward signal. As a consequence, an agent must know how to perform a multitude of tasks. However, when a new task is presented to an agent, we expect it to be able to reuse some of what it knows from previous tasks to rapidly learn that new task. We introduce a novel technique whereby policies for different a priori known tasks are combined into a Mixture-of-Experts model with an attention mechanism across a mix of frozen and unfrozen experts. The model learns when to attend to frozen task-specific experts when appropriate and learns new experts to handle novel situations. We work in an open-ended text-based environment in which the agent is tasked with behaving like different types of character roles and must rapidly learn behaviors associated with new character role types. We show that our agent both obtains more rewards in the zero-shot setting, and discovers these rewards with greater sample efficiency in the few-shot learning settings.