State-Only Imitation Learning for Dexterous Manipulation
This addresses the problem of learning from demonstrations in real-world settings like videos for robotics, where expert actions are hard to obtain, though it is incremental as it builds on existing imitation learning approaches.
The paper tackles the challenge of high sample complexity in dexterous manipulation by proposing state-only imitation learning, which uses an inverse dynamics model to predict actions from demonstrations without expert actions, achieving performance comparable to state-action methods and significantly outperforming reinforcement learning alone.
Modern model-free reinforcement learning methods have recently demonstrated impressive results on a number of problems. However, complex domains like dexterous manipulation remain a challenge due to the high sample complexity. To address this, current approaches employ expert demonstrations in the form of state-action pairs, which are difficult to obtain for real-world settings such as learning from videos. In this paper, we move toward a more realistic setting and explore state-only imitation learning. To tackle this setting, we train an inverse dynamics model and use it to predict actions for state-only demonstrations. The inverse dynamics model and the policy are trained jointly. Our method performs on par with state-action approaches and considerably outperforms RL alone. By not relying on expert actions, we are able to learn from demonstrations with different dynamics, morphologies, and objects. Videos available at https://people.eecs.berkeley.edu/~ilija/soil .