The Emergence of Abstract and Episodic Neurons in Episodic Meta-RL
This work provides insights into neural mechanisms in meta-RL, but it is incremental as it builds on existing methods to analyze neuron types without introducing new algorithms or broad performance gains.
The paper analyzes the reinstatement mechanism in episodic meta-RL to identify two neuron classes—Abstract and Episodic—that emerge in an agent's working memory during training on an episodic Harlow visual fixation task, revealing how knowledge is encoded across tasks and within specific episodes.
In this work, we analyze the reinstatement mechanism introduced by Ritter et al. (2018) to reveal two classes of neurons that emerge in the agent's working memory (an epLSTM cell) when trained using episodic meta-RL on an episodic variant of the Harlow visual fixation task. Specifically, Abstract neurons encode knowledge shared across tasks, while Episodic neurons carry information relevant for a specific episode's task.