SAFARI: Safe and Active Robot Imitation Learning with Imagination
This work is significant for robotic manipulation, as it aims to improve the safety and robustness of imitation learning agents by actively addressing out-of-distribution scenarios.
This paper addresses the problem of erroneous agent behavior in imitation learning when encountering out-of-distribution situations. The proposed SAFARI algorithm allows the agent to request additional human demonstrations during training and combines model-free acting with model-based planning at deployment to minimize state-distribution shift, leading to increased performance on manipulation tasks.
One of the main issues in Imitation Learning is the erroneous behavior of an agent when facing out-of-distribution situations, not covered by the set of demonstrations given by the expert. In this work, we tackle this problem by introducing a novel active learning and control algorithm, SAFARI. During training, it allows an agent to request further human demonstrations when these out-of-distribution situations are met. At deployment, it combines model-free acting using behavioural cloning with model-based planning to reduce state-distribution shift, using future state reconstruction as a test for state familiarity. We empirically demonstrate how this method increases the performance on a set of manipulation tasks with respect to passive Imitation Learning, by gathering more informative demonstrations and by minimizing state-distribution shift at test time. We also show how this method enables the agent to autonomously predict failure rapidly and safely.