Goal-Oriented Skill Abstraction for Offline Multi-Task Reinforcement Learning
This addresses the problem of inefficient knowledge transfer in offline multi-task RL for robotic manipulation, representing an incremental advancement through skill abstraction techniques.
The paper tackles the challenge of knowledge sharing in offline multi-task reinforcement learning by proposing GO-Skill, which extracts reusable skills through goal-oriented abstraction and vector quantization, resulting in improved performance on robotic manipulation tasks in the MetaWorld benchmark.
Offline multi-task reinforcement learning aims to learn a unified policy capable of solving multiple tasks using only pre-collected task-mixed datasets, without requiring any online interaction with the environment. However, it faces significant challenges in effectively sharing knowledge across tasks. Inspired by the efficient knowledge abstraction observed in human learning, we propose Goal-Oriented Skill Abstraction (GO-Skill), a novel approach designed to extract and utilize reusable skills to enhance knowledge transfer and task performance. Our approach uncovers reusable skills through a goal-oriented skill extraction process and leverages vector quantization to construct a discrete skill library. To mitigate class imbalances between broadly applicable and task-specific skills, we introduce a skill enhancement phase to refine the extracted skills. Furthermore, we integrate these skills using hierarchical policy learning, enabling the construction of a high-level policy that dynamically orchestrates discrete skills to accomplish specific tasks. Extensive experiments on diverse robotic manipulation tasks within the MetaWorld benchmark demonstrate the effectiveness and versatility of GO-Skill.