Cost Function Estimation Using Inverse Reinforcement Learning with Minimal Observations
This work addresses the challenge of cost function estimation for robotics or control systems with minimal observations, representing an incremental improvement over existing methods.
The paper tackles the problem of inferring optimal cost functions in continuous spaces using inverse reinforcement learning, achieving faster learning by tuning observation effectiveness and using informative trajectories from optimal control, with performance benefits demonstrated in simulated environments.
We present an iterative inverse reinforcement learning algorithm to infer optimal cost functions in continuous spaces. Based on a popular maximum entropy criteria, our approach iteratively finds a weight improvement step and proposes a method to find an appropriate step size that ensures learned cost function features remain similar to the demonstrated trajectory features. In contrast to similar approaches, our algorithm can individually tune the effectiveness of each observation for the partition function and does not need a large sample set, enabling faster learning. We generate sample trajectories by solving an optimal control problem instead of random sampling, leading to more informative trajectories. The performance of our method is compared to two state of the art algorithms to demonstrate its benefits in several simulated environments.