Inverse Reinforcement Learning with Missing Data
This addresses the issue of missing data in IRL for applications like vehicle tracking, but it is incremental as it builds on existing IRL methods with a focus on handling missing segments.
The paper tackles the problem of recovering an expert's reward function using inverse reinforcement learning when trajectory data has missing state-action pairs, such as intermittent GPS signals, by proposing an efficient algorithm that directly computes log-likelihood with incomplete data and outperforms conventional techniques in empirical evaluations.
We consider the problem of recovering an expert's reward function with inverse reinforcement learning (IRL) when there are missing/incomplete state-action pairs or observations in the demonstrated trajectories. This issue of missing trajectory data or information occurs in many situations, e.g., GPS signals from vehicles moving on a road network are intermittent. In this paper, we propose a tractable approach to directly compute the log-likelihood of demonstrated trajectories with incomplete/missing data. Our algorithm is efficient in handling a large number of missing segments in the demonstrated trajectories, as it performs the training with incomplete data by solving a sequence of systems of linear equations, and the number of such systems to be solved does not depend on the number of missing segments. Empirical evaluation on a real-world dataset shows that our training algorithm outperforms other conventional techniques.