On Approximate Dynamic Programming with Multivariate Splines for Adaptive Control
This work addresses adaptive control for time-varying systems, offering incremental improvements in performance and recovery speed.
The paper tackled adaptive control by developing a simplex dynamic programming (SDP) framework with multivariate splines and a modified RLSTD algorithm, showing increased performance in stability and learning rate compared to neural dynamic programming (NDP) and faster recovery in time-varying systems.
We define a SDP framework based on the RLSTD algorithm and multivariate simplex B-splines. We introduce a local forget factor capable of preserving the continuity of the simplex splines. This local forget factor is integrated with the RLSTD algorithm, resulting in a modified RLSTD algorithm that is capable of tracking time-varying systems. We present the results of two numerical experiments, one validating SDP and comparing it with NDP and another to show the advantages of the modified RLSTD algorithm over the original. While SDP requires more computations per time-step, the experiment shows that for the same amount of function approximator parameters, there is an increase in performance in terms of stability and learning rate compared to NDP. The second experiment shows that SDP in combination with the modified RLSTD algorithm allows for faster recovery compared to the original RLSTD algorithm when system parameters are altered, paving the way for an adaptive high-performance non-linear control method.