Generalized Maximum Causal Entropy for Inverse Reinforcement Learning
This work addresses a limitation in inverse reinforcement learning for applications like robotics and autonomous systems, but it is incremental as it builds upon existing maximum entropy models.
The authors tackled the problem of learning from demonstrated trajectories in inverse reinforcement learning by proposing a generalized maximum causal entropy model to better capture state network structure, resulting in improved performance in recovering reward functions and trajectories on real-world and grid-world datasets.
We consider the problem of learning from demonstrated trajectories with inverse reinforcement learning (IRL). Motivated by a limitation of the classical maximum entropy model in capturing the structure of the network of states, we propose an IRL model based on a generalized version of the causal entropy maximization problem, which allows us to generate a class of maximum entropy IRL models. Our generalized model has an advantage of being able to recover, in addition to a reward function, another expert's function that would (partially) capture the impact of the connecting structure of the states on experts' decisions. Empirical evaluation on a real-world dataset and a grid-world dataset shows that our generalized model outperforms the classical ones, in terms of recovering reward functions and demonstrated trajectories.