Intrinsic Rewards from Self-Organizing Feature Maps for Exploration in Reinforcement Learning
This addresses exploration challenges in reinforcement learning for agents, though it appears incremental as it builds on existing methods like ICM and RND.
The paper tackles the problem of exploration in reinforcement learning by introducing an intrinsic reward based on self-organizing feature maps, achieving human-level performance in the game Ordeal after a comparable number of training epochs to ICM.
We introduce an exploration bonus for deep reinforcement learning methods calculated using self-organising feature maps. Our method uses adaptive resonance theory (ART) providing online, unsupervised clustering to quantify the novelty of a state. This heuristic is used to add an intrinsic reward to the extrinsic reward signal for then to optimize the agent to maximize the sum of these two rewards. We find that this method was able to play the game Ordeal at a human level after a comparable number of training epochs to ICM arXiv:1705.05464. Agents augmented with RND arXiv:1810.12894 were unable to achieve the same level of performance in our space of hyperparameters.