Multi-task Maximum Entropy Inverse Reinforcement Learning
This addresses computational inefficiencies in multi-task IRL for robotics and AI applications, though it is incremental as it adapts an existing framework.
The paper tackles the problem of scaling multi-task inverse reinforcement learning to complex environments by formulating it within the Maximum Causal Entropy framework, achieving one-shot imitation learning in a gridworld where single-task methods require hundreds of demonstrations.
Multi-task Inverse Reinforcement Learning (IRL) is the problem of inferring multiple reward functions from expert demonstrations. Prior work, built on Bayesian IRL, is unable to scale to complex environments due to computational constraints. This paper contributes a formulation of multi-task IRL in the more computationally efficient Maximum Causal Entropy (MCE) IRL framework. Experiments show our approach can perform one-shot imitation learning in a gridworld environment that single-task IRL algorithms need hundreds of demonstrations to solve. We outline preliminary work using meta-learning to extend our method to the function approximator setting of modern MCE IRL algorithms. Evaluating on multi-task variants of common simulated robotics benchmarks, we discover serious limitations of these IRL algorithms, and conclude with suggestions for further work.