Memory Augmented Self-Play
This addresses the challenge of efficient exploration in reinforcement learning for AI agents, though it appears incremental by adding memory to an existing self-play framework.
The paper tackled the problem of slow exploration in unsupervised reinforcement learning by augmenting self-play with an external memory to store past experiences, resulting in faster environment exploration and outperforming no-memory self-play in pretraining.
Self-play is an unsupervised training procedure which enables the reinforcement learning agents to explore the environment without requiring any external rewards. We augment the self-play setting by providing an external memory where the agent can store experience from the previous tasks. This enables the agent to come up with more diverse self-play tasks resulting in faster exploration of the environment. The agent pretrained in the memory augmented self-play setting easily outperforms the agent pretrained in no-memory self-play setting.