On the Comparison of Reinforcement Learning and Adaptive Control for Linear Systems under Packet Loss and Uncertainty
For researchers in networked control systems, this paper highlights the trade-off between data-driven performance and model-based robustness, providing guidance on when to use reinforcement learning versus adaptive control.
This paper compares Adaptive Quantized Control (AQC) and Deep Deterministic Policy Gradient (DDPG) for uncertain linear systems with packet loss and dynamic switching. Results show DDPG achieves faster transient responses in training, but AQC demonstrates superior robustness under uncertainty and packet loss due to Lyapunov stability guarantees.
This paper presents a comparative study between Adaptive Quantized Control (AQC) and Deep Deterministic Policy Gradient (DDPG) reinforcement learning for uncertain linear systems with input quantization over communication channels subject to packet loss. The considered setting also includes dynamic switching from a nominal unstable system to a more unstable one during operation. The AQC is designed for unknown system dynamics using acknowledgment messages to compensate for packet losses, whereas the DDPG controller is trained using the nominal system model without acknowledgment messages. Numerical results show that the DDPG controller achieves faster transient responses and improved damping within its training environment. However, under model uncertainty, packet loss, and dynamic switching, the AQC consistently demonstrates superior robustness owing to its rigorous Lyapunov stability guarantees. These results highlight the trade-off between data-driven performance and model-based robustness, and provide insight into the applicability of reinforcement learning and adaptive control for networked uncertain systems.