ADAIL: Adaptive Adversarial Imitation Learning
This addresses a key challenge in robotic learning where reward functions are hard to obtain and policies are difficult to deploy across different dynamics, though it is incremental as it builds upon adversarial imitation learning.
The paper tackles the problem of transferring imitation learning policies between environments with varying dynamics by introducing ADAIL, which uses a dynamics embedding and domain-adversarial loss to learn adaptive policies from a single source domain, and demonstrates its effectiveness by outperforming baselines on simulated control tasks.
We present the ADaptive Adversarial Imitation Learning (ADAIL) algorithm for learning adaptive policies that can be transferred between environments of varying dynamics, by imitating a small number of demonstrations collected from a single source domain. This is an important problem in robotic learning because in real world scenarios 1) reward functions are hard to obtain, 2) learned policies from one domain are difficult to deploy in another due to varying source to target domain statistics, 3) collecting expert demonstrations in multiple environments where the dynamics are known and controlled is often infeasible. We address these constraints by building upon recent advances in adversarial imitation learning; we condition our policy on a learned dynamics embedding and we employ a domain-adversarial loss to learn a dynamics-invariant discriminator. The effectiveness of our method is demonstrated on simulated control tasks with varying environment dynamics and the learned adaptive agent outperforms several recent baselines.