Model-Based Inverse Reinforcement Learning from Visual Demonstrations
This work addresses a key challenge in robotics for enabling robots to learn from visual demonstrations, though it is incremental as it builds on existing model-based IRL methods.
The authors tackled the problem of scaling model-based inverse reinforcement learning to real robotic manipulation tasks with unknown dynamics by developing a gradient-based framework that uses a pre-trained visual dynamics model to learn cost functions from visual human demonstrations, achieving successful reproduction of demonstrated behavior on two hardware object manipulation tasks.
Scaling model-based inverse reinforcement learning (IRL) to real robotic manipulation tasks with unknown dynamics remains an open problem. The key challenges lie in learning good dynamics models, developing algorithms that scale to high-dimensional state-spaces and being able to learn from both visual and proprioceptive demonstrations. In this work, we present a gradient-based inverse reinforcement learning framework that utilizes a pre-trained visual dynamics model to learn cost functions when given only visual human demonstrations. The learned cost functions are then used to reproduce the demonstrated behavior via visual model predictive control. We evaluate our framework on hardware on two basic object manipulation tasks.