A Method to Improve the Performance of Reinforcement Learning Based on the Y Operator for a Class of Stochastic Differential Equation-Based Child-Mother Systems
This work addresses control performance issues in reinforcement learning for stochastic differential equation-based systems, representing an incremental advancement with a novel operator.
The paper tackles the problem of improving control performance in Actor-Critic reinforcement learning for systems governed by stochastic differential equations by introducing a novel Y operator, which integrates system stochasticity into the Critic network's loss function and reformulates the state-value function problem, resulting in enhanced performance over existing methods in linear and nonlinear numerical examples.
This paper introduces a novel operator, termed the Y operator, to elevate control performance in Actor-Critic(AC) based reinforcement learning for systems governed by stochastic differential equations(SDEs). The Y operator ingeniously integrates the stochasticity of a class of child-mother system into the Critic network's loss function, yielding substantial advancements in the control performance of RL algorithms.Additionally, the Y operator elegantly reformulates the challenge of solving partial differential equations for the state-value function into a parallel problem for the drift and diffusion functions within the system's SDEs.A rigorous mathematical proof confirms the operator's validity.This transformation enables the Y Operator-based Reinforcement Learning(YORL) framework to efficiently tackle optimal control problems in both model-based and data-driven systems.The superiority of YORL is demonstrated through linear and nonlinear numerical examples showing its enhanced performance over existing methods post convergence.