Online Adaptation for Enhancing Imitation Learning Policies
This addresses the issue of robustness in imitation learning for autonomous agents, though it appears incremental as it builds on existing methods.
The paper tackles the problem of imitation learning policies failing due to incomplete datasets or complex tasks by introducing online adaptation that combines pre-trained policy actions with expert experience. The result shows adapted agents outperform pure imitation learning and achieve reasonable performance even when the base policy fails catastrophically.
Imitation learning enables autonomous agents to learn from human examples, without the need for a reward signal. Still, if the provided dataset does not encapsulate the task correctly, or when the task is too complex to be modeled, such agents fail to reproduce the expert policy. We propose to recover from these failures through online adaptation. Our approach combines the action proposal coming from a pre-trained policy with relevant experience recorded by an expert. The combination results in an adapted action that closely follows the expert. Our experiments show that an adapted agent performs better than its pure imitation learning counterpart. Notably, adapted agents can achieve reasonable performance even when the base, non-adapted policy catastrophically fails.