Multi-fidelity Reinforcement Learning Control for Complex Dynamical Systems
This addresses the problem of expensive or sparse data collection in control tasks for scientific and engineering applications, offering an incremental improvement over existing methods.
The paper tackles the challenge of controlling instabilities in complex dynamical systems by proposing a multi-fidelity reinforcement learning framework that uses physics-based hybrid models corrected with limited high-fidelity data, achieving results that match many-query evaluations and outperform state-of-the-art baselines.
Controlling instabilities in complex dynamical systems is challenging in scientific and engineering applications. Deep reinforcement learning (DRL) has seen promising results for applications in different scientific applications. The many-query nature of control tasks requires multiple interactions with real environments of the underlying physics. However, it is usually sparse to collect from the experiments or expensive to simulate for complex dynamics. Alternatively, controlling surrogate modeling could mitigate the computational cost issue. However, a fast and accurate learning-based model by offline training makes it very hard to get accurate pointwise dynamics when the dynamics are chaotic. To bridge this gap, the current work proposes a multi-fidelity reinforcement learning (MFRL) framework that leverages differentiable hybrid models for control tasks, where a physics-based hybrid model is corrected by limited high-fidelity data. We also proposed a spectrum-based reward function for RL learning. The effect of the proposed framework is demonstrated on two complex dynamics in physics. The statistics of the MFRL control result match that computed from many-query evaluations of the high-fidelity environments and outperform other SOTA baselines.