Multitask Soft Option Learning
This addresses multitask reinforcement learning challenges for AI researchers, offering an incremental improvement over existing hierarchical methods.
The paper tackles the problem of training instability and forgetting in hierarchical multitask reinforcement learning by introducing Multitask Soft Option Learning (MSOL), which uses separate variational posteriors per task regularized by a shared prior, and demonstrates that it significantly outperforms hierarchical and flat transfer-learning baselines.
We present Multitask Soft Option Learning(MSOL), a hierarchical multitask framework based on Planning as Inference. MSOL extends the concept of options, using separate variational posteriors for each task, regularized by a shared prior. This ''soft'' version of options avoids several instabilities during training in a multitask setting, and provides a natural way to learn both intra-option policies and their terminations. Furthermore, it allows fine-tuning of options for new tasks without forgetting their learned policies, leading to faster training without reducing the expressiveness of the hierarchical policy. We demonstrate empirically that MSOL significantly outperforms both hierarchical and flat transfer-learning baselines.