Risk-Aware Active Inverse Reinforcement Learning
This work addresses the problem of efficient and safe policy learning for robots, though it is incremental as it builds on existing active IRL methods.
The authors tackled the problem of active learning from demonstration by proposing a risk-aware active inverse reinforcement learning algorithm that minimizes performance risk and focuses queries on high generalization error areas, showing it outperforms standard approaches on gridworld, simulated driving, and table setting tasks.
Active learning from demonstration allows a robot to query a human for specific types of input to achieve efficient learning. Existing work has explored a variety of active query strategies; however, to our knowledge, none of these strategies directly minimize the performance risk of the policy the robot is learning. Utilizing recent advances in performance bounds for inverse reinforcement learning, we propose a risk-aware active inverse reinforcement learning algorithm that focuses active queries on areas of the state space with the potential for large generalization error. We show that risk-aware active learning outperforms standard active IRL approaches on gridworld, simulated driving, and table setting tasks, while also providing a performance-based stopping criterion that allows a robot to know when it has received enough demonstrations to safely perform a task.