Evolutionarily-Curated Curriculum Learning for Deep Reinforcement Learning Agents
This addresses training efficiency and generalization for deep reinforcement learning agents, but it is incremental as it builds on existing evolutionary and curriculum learning methods.
The paper tackles the problem of training deep reinforcement learning agents more efficiently and with better generalization by introducing an evolutionarily-curated curriculum of maps, showing that this method expedites training and improves generalization compared to undirected sampling in a custom game.
In this paper we propose a new training loop for deep reinforcement learning agents with an evolutionary generator. Evolutionary procedural content generation has been used in the creation of maps and levels for games before. Our system incorporates an evolutionary map generator to construct a training curriculum that is evolved to maximize loss within the state-of-the-art Double Dueling Deep Q Network architecture with prioritized replay. We present a case-study in which we prove the efficacy of our new method on a game with a discrete, large action space we made called Attackers and Defenders. Our results demonstrate that training on an evolutionarily-curated curriculum (directed sampling) of maps both expedites training and improves generalization when compared to a network trained on an undirected sampling of maps.