Data-Assimilated Model-Based Reinforcement Learning for Partially Observed Chaotic Flows
This addresses the problem of controlling chaotic flows in energy and transport sectors, offering a novel integration of methods for partial observability.
The paper tackles controlling turbulent flows with partial observability by proposing a data-assimilated model-based reinforcement learning framework, which successfully stabilizes a chaotic flow from noisy measurements as demonstrated on the Kuramoto-Sivashinsky equation.
The goal of many applications in energy and transport sectors is to control turbulent flows. However, because of chaotic dynamics and high dimensionality, the control of turbulent flows is exceedingly difficult. Model-free reinforcement learning (RL) methods can discover optimal control policies by interacting with the environment, but they require full state information, which is often unavailable in experimental settings. We propose a data-assimilated model-based RL (DA-MBRL) framework for systems with partial observability and noisy measurements. Our framework employs a control-aware Echo State Network for data-driven prediction of the dynamics, and integrates data assimilation with an Ensemble Kalman Filter for real-time state estimation. An off-policy actor-critic algorithm is employed to learn optimal control strategies from state estimates. The framework is tested on the Kuramoto-Sivashinsky equation, demonstrating its effectiveness in stabilizing a spatiotemporally chaotic flow from noisy and partial measurements.