SD2AIL: Adversarial Imitation Learning from Synthetic Demonstrations via Diffusion Models
This work addresses the problem of limited expert data in imitation learning for robotics simulation tasks, offering an incremental improvement over existing methods.
The paper tackles the challenge of collecting expert demonstrations for adversarial imitation learning by generating synthetic demonstrations using diffusion models and introducing a prioritized replay strategy, achieving an average return of 3441 in the Hopper task, surpassing the state-of-the-art by 89.
Adversarial Imitation Learning (AIL) is a dominant framework in imitation learning that infers rewards from expert demonstrations to guide policy optimization. Although providing more expert demonstrations typically leads to improved performance and greater stability, collecting such demonstrations can be challenging in certain scenarios. Inspired by the success of diffusion models in data generation, we propose SD2AIL, which utilizes synthetic demonstrations via diffusion models. We first employ a diffusion model in the discriminator to generate synthetic demonstrations as pseudo-expert data that augment the expert demonstrations. To selectively replay the most valuable demonstrations from the large pool of (pseudo-) expert demonstrations, we further introduce a prioritized expert demonstration replay strategy (PEDR). The experimental results on simulation tasks demonstrate the effectiveness and robustness of our method. In particular, in the Hopper task, our method achieves an average return of 3441, surpassing the state-of-the-art method by 89. Our code will be available at https://github.com/positron-lpc/SD2AIL.