Clustered Reinforcement Learning
This addresses exploration challenges in RL for environments with large state spaces or sparse rewards, but it appears incremental as it builds on existing methods by incorporating clustering.
The paper tackles the problem of inefficient exploration in reinforcement learning by proposing a clustered RL framework that uses clustering to provide bonus rewards based on novelty and quality in neighboring areas, resulting in outperforming state-of-the-art methods on continuous control tasks and Atari games.
Exploration strategy design is one of the challenging problems in reinforcement learning~(RL), especially when the environment contains a large state space or sparse rewards. During exploration, the agent tries to discover novel areas or high reward~(quality) areas. In most existing methods, the novelty and quality in the neighboring area of the current state are not well utilized to guide the exploration of the agent. To tackle this problem, we propose a novel RL framework, called \underline{c}lustered \underline{r}einforcement \underline{l}earning~(CRL), for efficient exploration in RL. CRL adopts clustering to divide the collected states into several clusters, based on which a bonus reward reflecting both novelty and quality in the neighboring area~(cluster) of the current state is given to the agent. Experiments on a continuous control task and several \emph{Atari 2600} games show that CRL can outperform other state-of-the-art methods to achieve the best performance in most cases.