Options Discovery with Budgeted Reinforcement Learning
This addresses the challenge of automating option discovery in RL, which is incremental as it builds on existing hierarchical policy methods.
The paper tackles the problem of automatically discovering hierarchical policies (options) in reinforcement learning, which traditionally rely on predefined sets, by introducing the Budgeted Option Neural Network (BONN) model and demonstrates its effectiveness with quantitative and qualitative results on classical RL problems.
We consider the problem of learning hierarchical policies for Reinforcement Learning able to discover options, an option corresponding to a sub-policy over a set of primitive actions. Different models have been proposed during the last decade that usually rely on a predefined set of options. We specifically address the problem of automatically discovering options in decision processes. We describe a new learning model called Budgeted Option Neural Network (BONN) able to discover options based on a budgeted learning objective. The BONN model is evaluated on different classical RL problems, demonstrating both quantitative and qualitative interesting results.