Learning Task Agnostic Skills with Data-driven Guidance
This addresses the challenge of increasing autonomy in reinforcement learning for agents by improving skill discovery without manual reward design, though it appears incremental as it builds on existing methods with a specific guidance mechanism.
The paper tackles the problem of agents learning useless behaviors in task-agnostic skill discovery due to lack of task-specific rewards, by proposing a framework that guides skill discovery towards expert-visited states using a learned state projection, resulting in more useful behaviors as demonstrated in various RL tasks.
To increase autonomy in reinforcement learning, agents need to learn useful behaviours without reliance on manually designed reward functions. To that end, skill discovery methods have been used to learn the intrinsic options available to an agent using task-agnostic objectives. However, without the guidance of task-specific rewards, emergent behaviours are generally useless due to the under-constrained problem of skill discovery in complex and high-dimensional spaces. This paper proposes a framework for guiding the skill discovery towards the subset of expert-visited states using a learned state projection. We apply our method in various reinforcement learning (RL) tasks and show that such a projection results in more useful behaviours.