Model-free optimal control of discrete-time systems with additive and multiplicative noises
This addresses control design for stochastic systems, offering a model-free approach that could benefit robotics or autonomous systems, but it appears incremental as it builds on existing policy iteration methods.
The paper tackles the optimal control problem for discrete-time stochastic systems with additive and multiplicative noises by proposing a model-free reinforcement learning algorithm that learns the optimal policy without system knowledge, proving convergence and demonstrating in a numerical example that it outperforms other policy iteration algorithms.
This paper investigates the optimal control problem for a class of discrete-time stochastic systems subject to additive and multiplicative noises. A stochastic Lyapunov equation and a stochastic algebra Riccati equation are established for the existence of the optimal admissible control policy. A model-free reinforcement learning algorithm is proposed to learn the optimal admissible control policy using the data of the system states and inputs without requiring any knowledge of the system matrices. It is proven that the learning algorithm converges to the optimal admissible control policy. The implementation of the model-free algorithm is based on batch least squares and numerical average. The proposed algorithm is illustrated through a numerical example, which shows our algorithm outperforms other policy iteration algorithms.