Dyna-AIL : Adversarial Imitation Learning by Planning
This addresses the efficiency issue in imitation learning for control tasks, but it appears incremental as it builds on existing adversarial and Dyna frameworks.
The paper tackles the problem of adversarial imitation learning requiring many environment interactions by proposing an end-to-end differentiable algorithm that combines model-based planning and model-free learning, resulting in convergence to an optimal policy with fewer interactions than state-of-the-art methods.
Adversarial methods for imitation learning have been shown to perform well on various control tasks. However, they require a large number of environment interactions for convergence. In this paper, we propose an end-to-end differentiable adversarial imitation learning algorithm in a Dyna-like framework for switching between model-based planning and model-free learning from expert data. Our results on both discrete and continuous environments show that our approach of using model-based planning along with model-free learning converges to an optimal policy with fewer number of environment interactions in comparison to the state-of-the-art learning methods.