Grounding Hierarchical Reinforcement Learning Models for Knowledge Transfer
This work addresses a gap in deep learning for RL by extending it from model-free to model-based approaches, which is incremental as it builds on existing deep RL methods.
The paper tackles the problem of learning abstract hierarchical representations in model-based reinforcement learning by grounding them in sensorimotor interaction, enabling agents to acquire arbitrarily abstract environmental models.
Methods of deep machine learning enable to to reuse low-level representations efficiently for generating more abstract high-level representations. Originally, deep learning has been applied passively (e.g., for classification purposes). Recently, it has been extended to estimate the value of actions for autonomous agents within the framework of reinforcement learning (RL). Explicit models of the environment can be learned to augment such a value function. Although "flat" connectionist methods have already been used for model-based RL, up to now, only model-free variants of RL have been equipped with methods from deep learning. We propose a variant of deep model-based RL that enables an agent to learn arbitrarily abstract hierarchical representations of its environment. In this paper, we present research on how such hierarchical representations can be grounded in sensorimotor interaction between an agent and its environment.