Weighted Maximum Entropy Inverse Reinforcement Learning
This is an incremental improvement for robotics and autonomous systems that need to learn from expert demonstrations.
The paper tackles the problem of inverse reinforcement learning and imitation learning by proposing a weighted maximum entropy framework to better capture expert stochasticity, with numerical experiments showing it outperforms prior algorithms.
We study inverse reinforcement learning (IRL) and imitation learning (IM), the problems of recovering a reward or policy function from expert's demonstrated trajectories. We propose a new way to improve the learning process by adding a weight function to the maximum entropy framework, with the motivation of having the ability to learn and recover the stochasticity (or the bounded rationality) of the expert policy. Our framework and algorithms allow to learn both a reward (or policy) function and the structure of the entropy terms added to the Markov Decision Processes, thus enhancing the learning procedure. Our numerical experiments using human and simulated demonstrations and with discrete and continuous IRL/IM tasks show that our approach outperforms prior algorithms.