Actor-Identifier-Critic Reinforcement Learning for Adaptive Model-Free Optimal Control of Nonlinear Systems with Stochastic Packet Dropouts
For control engineers dealing with nonlinear systems under communication constraints, this work addresses the challenge of packet dropouts without requiring an accurate system model.
This paper proposes an Actor-Identifier-Critic (AIC) controller for model-free tracking control of nonlinear systems with stochastic packet dropouts in both controller-to-actuator and sensor-to-controller channels. The method is demonstrated on two nonlinear systems and a power system stability case study, showing effective handling of packet dropouts.
Packet dropouts in control systems poses a critical challenge, as it can significantly compromise system performance and stability. In these conditions, classical controllers often struggle to deliver effective control, as they rely on accurate system models, which may not always be available. This paper proposes a novel Actor-Identifier-Critic~(AIC) controller to address model-free tracking control of nonlinear systems in the presence of packet dropouts in both the controller-to-actuator and sensor-to-controller channels. Using an identifier to learn the system dynamics, the proposed controller is able to handle packet dropouts in the communication link and facilitate gradient propagation from the critic to the actor within a model-free control framework. The performance of the proposed method is demonstrated on two nonlinear SIMO and MIMO systems and a case study on power system stability subject to stochastic packet dropouts.