Nonuniqueness and Convergence to Equivalent Solutions in Observer-based Inverse Reinforcement Learning
This addresses a key challenge in online, real-time IRL for robotics or control systems, though it is incremental as it builds on prior offline methods.
The paper tackles the problem of nonuniqueness in deterministic inverse reinforcement learning (IRL) by developing a regularized history stack observer that converges to approximately equivalent solutions, with simulation results demonstrating its effectiveness.
A key challenge in solving the deterministic inverse reinforcement learning (IRL) problem online and in real-time is the existence of multiple solutions. Nonuniqueness necessitates the study of the notion of equivalent solutions, i.e., solutions that result in a different cost functional but same feedback matrix, and convergence to such solutions. While offline algorithms that result in convergence to equivalent solutions have been developed in the literature, online, real-time techniques that address nonuniqueness are not available. In this paper, a regularized history stack observer that converges to approximately equivalent solutions of the IRL problem is developed. Novel data-richness conditions are developed to facilitate the analysis and simulation results are provided to demonstrate the effectiveness of the developed technique.