Automata-Guided Hierarchical Reinforcement Learning for Skill Composition
This work addresses the need for more adaptable and reusable skills in reinforcement learning, though it appears incremental as it builds on existing hierarchical RL and formal methods.
The paper tackles the problem of poor skill generalization in reinforcement learning by introducing a framework that combines formal methods with RL, enabling task specification via logical expressions and skill composition with minimal additional exploration. The method was evaluated in a grid world simulation and a kitchen environment in AI2Thor.
Skills learned through (deep) reinforcement learning often generalizes poorly across domains and re-training is necessary when presented with a new task. We present a framework that combines techniques in \textit{formal methods} with \textit{reinforcement learning} (RL). The methods we provide allows for convenient specification of tasks with logical expressions, learns hierarchical policies (meta-controller and low-level controllers) with well-defined intrinsic rewards, and construct new skills from existing ones with little to no additional exploration. We evaluate the proposed methods in a simple grid world simulation as well as a more complicated kitchen environment in AI2Thor