SeMAIL: Eliminating Distractors in Visual Imitation via Separated Models
This addresses a key bottleneck in visual imitation learning for robotics and autonomous systems, offering an incremental improvement over existing methods.
The paper tackles the problem of task-irrelevant distractors in model-based imitation learning, proposing SeMAIL to decouple environment dynamics and achieve near-expert performance on visual control tasks with complex observations and varying backgrounds.
Model-based imitation learning (MBIL) is a popular reinforcement learning method that improves sample efficiency on high-dimension input sources, such as images and videos. Following the convention of MBIL research, existing algorithms are highly deceptive by task-irrelevant information, especially moving distractors in videos. To tackle this problem, we propose a new algorithm - named Separated Model-based Adversarial Imitation Learning (SeMAIL) - decoupling the environment dynamics into two parts by task-relevant dependency, which is determined by agent actions, and training separately. In this way, the agent can imagine its trajectories and imitate the expert behavior efficiently in task-relevant state space. Our method achieves near-expert performance on various visual control tasks with complex observations and the more challenging tasks with different backgrounds from expert observations.