Model-based reinforcement learning for infinite-horizon approximate optimal tracking
For control engineers, it relaxes the persistence of excitation condition in optimal tracking, but the approach is incremental.
The paper presents a model-based reinforcement learning approach for infinite-horizon optimal tracking in nonlinear systems with unknown dynamics, achieving convergence to near-optimal policy without persistence of excitation. Simulations validate the method.
This paper provides an approximate online adaptive solution to the infinite-horizon optimal tracking problem for control-affine continuous-time nonlinear systems with unknown drift dynamics. Model-based reinforcement learning is used to relax the persistence of excitation condition. Model-based reinforcement learning is implemented using a concurrent learning-based system identifier to simulate experience by evaluating the Bellman error over unexplored areas of the state space. Tracking of the desired trajectory and convergence of the developed policy to a neighborhood of the optimal policy are established via Lyapunov-based stability analysis. Simulation results demonstrate the effectiveness of the developed technique.