Hierarchical Few-Shot Imitation with Skill Transition Models
This addresses the problem of task generalization for autonomous agents, representing an incremental advance over prior skill learning methods.
The paper tackles the challenge of generalizing to unseen tasks in long-horizon problems by introducing FIST, an algorithm that extracts skills from offline data and uses few-shot demonstrations, achieving substantial performance improvements in navigation and robotic manipulation experiments.
A desirable property of autonomous agents is the ability to both solve long-horizon problems and generalize to unseen tasks. Recent advances in data-driven skill learning have shown that extracting behavioral priors from offline data can enable agents to solve challenging long-horizon tasks with reinforcement learning. However, generalization to tasks unseen during behavioral prior training remains an outstanding challenge. To this end, we present Few-shot Imitation with Skill Transition Models (FIST), an algorithm that extracts skills from offline data and utilizes them to generalize to unseen tasks given a few downstream demonstrations. FIST learns an inverse skill dynamics model, a distance function, and utilizes a semi-parametric approach for imitation. We show that FIST is capable of generalizing to new tasks and substantially outperforms prior baselines in navigation experiments requiring traversing unseen parts of a large maze and 7-DoF robotic arm experiments requiring manipulating previously unseen objects in a kitchen.