MGHRL: Meta Goal-generation for Hierarchical Reinforcement Learning
This addresses a bottleneck in meta-RL for robotics, offering an incremental improvement over existing methods.
The paper tackles the inefficiency of meta-RL methods in wide task distributions by proposing MGHRL, which generates high-level meta strategies over subgoals instead of primitive actions, resulting in more efficient and generalized meta-learning in simulated robotics environments.
Most meta reinforcement learning (meta-RL) methods learn to adapt to new tasks by directly optimizing the parameters of policies over primitive action space. Such algorithms work well in tasks with relatively slight difference. However, when the task distribution becomes wider, it would be quite inefficient to directly learn such a meta-policy. In this paper, we propose a new meta-RL algorithm called Meta Goal-generation for Hierarchical RL (MGHRL). Instead of directly generating policies over primitive action space for new tasks, MGHRL learns to generate high-level meta strategies over subgoals given past experience and leaves the rest of how to achieve subgoals as independent RL subtasks. Our empirical results on several challenging simulated robotics environments show that our method enables more efficient and generalized meta-learning from past experience.