Hyperbolic Embeddings for Learning Options in Hierarchical Reinforcement Learning
This addresses the challenge of breaking down large tasks into sub-tasks for reinforcement learning agents, though it appears incremental as it builds on existing hyperbolic embedding and options framework methods.
The paper tackles the problem of autonomously discovering meaningful sub-tasks in hierarchical reinforcement learning by embedding states into hyperbolic space to enforce a global topology, and demonstrates empirically that this improves sub-task learning in both discrete and continuous domains.
Hierarchical reinforcement learning deals with the problem of breaking down large tasks into meaningful sub-tasks. Autonomous discovery of these sub-tasks has remained a challenging problem. We propose a novel method of learning sub-tasks by combining paradigms of routing in computer networks and graph based skill discovery within the options framework to define meaningful sub-goals. We apply the recent advancements of learning embeddings using Riemannian optimisation in the hyperbolic space to embed the state set into the hyperbolic space and create a model of the environment. In doing so we enforce a global topology on the states and are able to exploit this topology to learn meaningful sub-tasks. We demonstrate empirically, both in discrete and continuous domains, how these embeddings can improve the learning of meaningful sub-tasks.