Optimal Options for Multi-Task Reinforcement Learning Under Time Constraints
This work addresses a key challenge in reinforcement learning for agents handling related tasks with limited time, though it is incremental in nature.
The paper tackles the problem of autonomously learning useful options in multi-task reinforcement learning under time constraints, showing that optimal options vary with learning time budgets and outperform existing heuristics.
Reinforcement learning can greatly benefit from the use of options as a way of encoding recurring behaviours and to foster exploration. An important open problem is how can an agent autonomously learn useful options when solving particular distributions of related tasks. We investigate some of the conditions that influence optimality of options, in settings where agents have a limited time budget for learning each task and the task distribution might involve problems with different levels of similarity. We directly search for optimal option sets and show that the discovered options significantly differ depending on factors such as the available learning time budget and that the found options outperform popular option-generation heuristics.