Extraneousness-Aware Imitation Learning
This addresses a common real-world problem in robotics where demonstrations contain extraneous but structured noise, offering a practical solution for learning from imperfect data.
The paper tackles imitation learning from demonstrations containing task-irrelevant but locally consistent segments (e.g., wiping sweat while cooking), introducing Extraneousness-Aware Imitation Learning (EIL) to learn visuomotor policies from such noisy data. It shows EIL outperforms baselines and achieves policies comparable to those trained with perfect demonstrations on simulated and real-world robot tasks.
Visual imitation learning provides an effective framework to learn skills from demonstrations. However, the quality of the provided demonstrations usually significantly affects the ability of an agent to acquire desired skills. Therefore, the standard visual imitation learning assumes near-optimal demonstrations, which are expensive or sometimes prohibitive to collect. Previous works propose to learn from noisy demonstrations; however, the noise is usually assumed to follow a context-independent distribution such as a uniform or gaussian distribution. In this paper, we consider another crucial yet underexplored setting -- imitation learning with task-irrelevant yet locally consistent segments in the demonstrations (e.g., wiping sweat while cutting potatoes in a cooking tutorial). We argue that such noise is common in real world data and term them "extraneous" segments. To tackle this problem, we introduce Extraneousness-Aware Imitation Learning (EIL), a self-supervised approach that learns visuomotor policies from third-person demonstrations with extraneous subsequences. EIL learns action-conditioned observation embeddings in a self-supervised manner and retrieves task-relevant observations across visual demonstrations while excluding the extraneous ones. Experimental results show that EIL outperforms strong baselines and achieves comparable policies to those trained with perfect demonstration on both simulated and real-world robot control tasks. The project page can be found at https://sites.google.com/view/eil-website.