Curious Hierarchical Actor-Critic Reinforcement Learning
This work addresses the lack of combined hierarchical and curiosity methods in reinforcement learning, offering a novel approach that enhances performance in complex environments, though it is incremental as it builds on existing hierarchical actor-critic techniques.
The paper tackled the problem of combining hierarchical reinforcement learning with curiosity-driven exploration to improve learning performance in continuous-space environments, demonstrating that curiosity can more than double learning performance and success rates in most benchmarking problems.
Hierarchical abstraction and curiosity-driven exploration are two common paradigms in current reinforcement learning approaches to break down difficult problems into a sequence of simpler ones and to overcome reward sparsity. However, there is a lack of approaches that combine these paradigms, and it is currently unknown whether curiosity also helps to perform the hierarchical abstraction. As a novelty and scientific contribution, we tackle this issue and develop a method that combines hierarchical reinforcement learning with curiosity. Herein, we extend a contemporary hierarchical actor-critic approach with a forward model to develop a hierarchical notion of curiosity. We demonstrate in several continuous-space environments that curiosity can more than double the learning performance and success rates for most of the investigated benchmarking problems. We also provide our source code and a supplementary video.