Get Back Here: Robust Imitation by Return-to-Distribution Planning
This addresses robustness in imitation learning for robotics, but it is incremental as it builds on existing methods like behavior cloning and planning.
The paper tackles the problem of imitation learning when expert data is collected in a different environment than deployment, by combining behavior cloning with a planner that returns the agent to expert-visited states, and demonstrates robustness to varied initial states and noisy dynamics in robotic manipulation simulations.
We consider the Imitation Learning (IL) setup where expert data are not collected on the actual deployment environment but on a different version. To address the resulting distribution shift, we combine behavior cloning (BC) with a planner that is tasked to bring the agent back to states visited by the expert whenever the agent deviates from the demonstration distribution. The resulting algorithm, POIR, can be trained offline, and leverages online interactions to efficiently fine-tune its planner to improve performance over time. We test POIR on a variety of human-generated manipulation demonstrations in a realistic robotic manipulation simulator and show robustness of the learned policy to different initial state distributions and noisy dynamics.