Imitation Learning from Imperfect Demonstration
This addresses the challenge of costly optimal demonstration collection in imitation learning, though it is incremental as it builds on existing IL frameworks.
The paper tackles the problem of imitation learning from imperfect demonstrations by proposing confidence-based methods, achieving significant performance improvements both theoretically and empirically.
Imitation learning (IL) aims to learn an optimal policy from demonstrations. However, such demonstrations are often imperfect since collecting optimal ones is costly. To effectively learn from imperfect demonstrations, we propose a novel approach that utilizes confidence scores, which describe the quality of demonstrations. More specifically, we propose two confidence-based IL methods, namely two-step importance weighting IL (2IWIL) and generative adversarial IL with imperfect demonstration and confidence (IC-GAIL). We show that confidence scores given only to a small portion of sub-optimal demonstrations significantly improve the performance of IL both theoretically and empirically.