Continuous Inverse Optimal Control with Locally Optimal Examples
This addresses the challenge of learning reward functions from suboptimal demonstrations in large, continuous domains, which is incremental but improves applicability over existing methods.
The paper tackles the problem of inverse optimal control in continuous domains where computing full policies is impractical, introducing a probabilistic algorithm that scales with task dimensionality and can learn from locally optimal demonstrations. The result is a method that works with examples unsuitable for prior approaches.
Inverse optimal control, also known as inverse reinforcement learning, is the problem of recovering an unknown reward function in a Markov decision process from expert demonstrations of the optimal policy. We introduce a probabilistic inverse optimal control algorithm that scales gracefully with task dimensionality, and is suitable for large, continuous domains where even computing a full policy is impractical. By using a local approximation of the reward function, our method can also drop the assumption that the demonstrations are globally optimal, requiring only local optimality. This allows it to learn from examples that are unsuitable for prior methods.