DIDA: Denoised Imitation Learning based on Domain Adaptation
This addresses the challenge of robust imitation learning from low-quality data, which is common in real-world applications, but it appears incremental as it builds on domain adaptation techniques.
The paper tackles the problem of learning from noisy demonstrations in imitation learning, where existing methods fail, and proposes DIDA, which uses domain adaptation to learn task-related representations, achieving successful imitation on MuJoCo tasks with various noise types and outperforming most baselines.
Imitating skills from low-quality datasets, such as sub-optimal demonstrations and observations with distractors, is common in real-world applications. In this work, we focus on the problem of Learning from Noisy Demonstrations (LND), where the imitator is required to learn from data with noise that often occurs during the processes of data collection or transmission. Previous IL methods improve the robustness of learned policies by injecting an adversarially learned Gaussian noise into pure expert data or utilizing additional ranking information, but they may fail in the LND setting. To alleviate the above problems, we propose Denoised Imitation learning based on Domain Adaptation (DIDA), which designs two discriminators to distinguish the noise level and expertise level of data, facilitating a feature encoder to learn task-related but domain-agnostic representations. Experiment results on MuJoCo demonstrate that DIDA can successfully handle challenging imitation tasks from demonstrations with various types of noise, outperforming most baseline methods.