Multi-task Hierarchical Adversarial Inverse Reinforcement Learning
This addresses data efficiency and performance issues in multi-task imitation learning for general-purpose robots, representing a novel method for a known bottleneck.
The paper tackles the problem of low data efficiency and poor performance in Multi-task Imitation Learning (MIL) for complex tasks by developing MH-AIRL, a method that learns hierarchically-structured policies, resulting in superior performance and transferability compared to state-of-the-art baselines.
Multi-task Imitation Learning (MIL) aims to train a policy capable of performing a distribution of tasks based on multi-task expert demonstrations, which is essential for general-purpose robots. Existing MIL algorithms suffer from low data efficiency and poor performance on complex long-horizontal tasks. We develop Multi-task Hierarchical Adversarial Inverse Reinforcement Learning (MH-AIRL) to learn hierarchically-structured multi-task policies, which is more beneficial for compositional tasks with long horizons and has higher expert data efficiency through identifying and transferring reusable basic skills across tasks. To realize this, MH-AIRL effectively synthesizes context-based multi-task learning, AIRL (an IL approach), and hierarchical policy learning. Further, MH-AIRL can be adopted to demonstrations without the task or skill annotations (i.e., state-action pairs only) which are more accessible in practice. Theoretical justifications are provided for each module of MH-AIRL, and evaluations on challenging multi-task settings demonstrate superior performance and transferability of the multi-task policies learned with MH-AIRL as compared to SOTA MIL baselines.