Been There, Done That: Meta-Learning with Episodic Recall
This addresses the issue of task forgetting in meta-learning for AI agents, enabling better reuse of learned solutions in repetitive scenarios, though it is incremental as it builds on existing memory methods.
The paper tackled the problem of meta-learning agents forgetting previously learned tasks when new ones begin, which hinders exploitation in repetitious environments, and developed an architecture combining LSTM with neural episodic memory, showing improved performance in five environments with reoccurring tasks.
Meta-learning agents excel at rapidly learning new tasks from open-ended task distributions; yet, they forget what they learn about each task as soon as the next begins. When tasks reoccur - as they do in natural environments - metalearning agents must explore again instead of immediately exploiting previously discovered solutions. We propose a formalism for generating open-ended yet repetitious environments, then develop a meta-learning architecture for solving these environments. This architecture melds the standard LSTM working memory with a differentiable neural episodic memory. We explore the capabilities of agents with this episodic LSTM in five meta-learning environments with reoccurring tasks, ranging from bandits to navigation and stochastic sequential decision problems.