MEGA-DAgger: Imitation Learning with Multiple Imperfect Experts
This addresses a practical limitation in autonomous systems where perfect experts are unavailable, though it is incremental as it builds on existing DAgger variants.
The paper tackles the problem of imitation learning with multiple imperfect experts, proposing MEGA-DAgger to filter unsafe demonstrations and resolve label conflicts, resulting in a policy that outperforms experts and state-of-the-art methods in autonomous racing scenarios.
Imitation learning has been widely applied to various autonomous systems thanks to recent development in interactive algorithms that address covariate shift and compounding errors induced by traditional approaches like behavior cloning. However, existing interactive imitation learning methods assume access to one perfect expert. Whereas in reality, it is more likely to have multiple imperfect experts instead. In this paper, we propose MEGA-DAgger, a new DAgger variant that is suitable for interactive learning with multiple imperfect experts. First, unsafe demonstrations are filtered while aggregating the training data, so the imperfect demonstrations have little influence when training the novice policy. Next, experts are evaluated and compared on scenarios-specific metrics to resolve the conflicted labels among experts. Through experiments in autonomous racing scenarios, we demonstrate that policy learned using MEGA-DAgger can outperform both experts and policies learned using the state-of-the-art interactive imitation learning algorithms such as Human-Gated DAgger. The supplementary video can be found at \url{https://youtu.be/wPCht31MHrw}.