Imitation by Predicting Observations
This addresses the problem of learning from demonstrations when actions are unavailable, which is incremental but practical for real-world robotics and AI applications.
The paper tackles imitation learning from observations without access to expert actions, presenting a method called FORM that achieves performance comparable to experts on continuous control tasks and shows robustness to task-irrelevant features.
Imitation learning enables agents to reuse and adapt the hard-won expertise of others, offering a solution to several key challenges in learning behavior. Although it is easy to observe behavior in the real-world, the underlying actions may not be accessible. We present a new method for imitation solely from observations that achieves comparable performance to experts on challenging continuous control tasks while also exhibiting robustness in the presence of observations unrelated to the task. Our method, which we call FORM (for "Future Observation Reward Model") is derived from an inverse RL objective and imitates using a model of expert behavior learned by generative modelling of the expert's observations, without needing ground truth actions. We show that FORM performs comparably to a strong baseline IRL method (GAIL) on the DeepMind Control Suite benchmark, while outperforming GAIL in the presence of task-irrelevant features.