Model-based deep reinforcement learning for accelerated learning from flow simulations
This addresses the bottleneck of slow simulations for real-world flow control applications, though it is incremental as it builds on existing model-based methods.
The paper tackles the high computational cost of flow simulations in deep reinforcement learning for flow control by using model-based reinforcement learning with an ensemble of environment models, reducing training time by up to 85% for the fluidic pinball test case.
In recent years, deep reinforcement learning has emerged as a technique to solve closed-loop flow control problems. Employing simulation-based environments in reinforcement learning enables a priori end-to-end optimization of the control system, provides a virtual testbed for safety-critical control applications, and allows to gain a deep understanding of the control mechanisms. While reinforcement learning has been applied successfully in a number of rather simple flow control benchmarks, a major bottleneck toward real-world applications is the high computational cost and turnaround time of flow simulations. In this contribution, we demonstrate the benefits of model-based reinforcement learning for flow control applications. Specifically, we optimize the policy by alternating between trajectories sampled from flow simulations and trajectories sampled from an ensemble of environment models. The model-based learning reduces the overall training time by up to $85\%$ for the fluidic pinball test case. Even larger savings are expected for more demanding flow simulations.