Model-Free Reinforcement Learning Control for Resilient Cyber-Physical Systems
For cyber-physical systems security, this work provides a comparative analysis of RL reward designs for resilience, but is incremental as it applies existing methods to a specific problem.
This paper compares model-free RL controllers on a nonlinear system under cyberattacks, finding that Lyapunov rewards provide the best resilience with low tracking error, and Proximal Policy Optimization outperforms Deep Deterministic Policy Gradient with significant reduction in KPI variance.
This paper compares the performance of model-free controllers on a nonlinear system under cyberattacks, including false data injection and denial-of-service attacks. Four RL reward types are analyzed for accuracy, cost, and resilience. Results show that the Lyapunov reward offers the best resilience with low tracking error. Exponential mode also provides good trade-offs with acceptable resilience under moderate training conditions. Progressive and linear rewards converge faster but are less robust. RL-MPCs show strong steady-state resilience but require longer training times; RL-PID controllers are faster with significantly less training time. Proximal Policy Optimization outperforms Deep Deterministic Policy Gradient with a significant reduction in KPI variance. This study serves to highlight how well-designed RL rewards can improve performance and resilience against cyber threats.