Principled Option Learning in Markov Decision Processes
This work addresses the need for principled methods in option learning for planning efficiency, though it appears incremental as it builds on heuristic approaches.
The paper tackles the problem of autonomously discovering useful options in Markov Decision Processes by proposing a mathematical characterization using information theory, resulting in an algorithm that outputs a useful set of options and is illustrated in simulation.
It is well known that options can make planning more efficient, among their many benefits. Thus far, algorithms for autonomously discovering a set of useful options were heuristic. Naturally, a principled way of finding a set of useful options may be more promising and insightful. In this paper we suggest a mathematical characterization of good sets of options using tools from information theory. This characterization enables us to find conditions for a set of options to be optimal and an algorithm that outputs a useful set of options and illustrate the proposed algorithm in simulation.