Interpretable and Efficient Data-driven Discovery and Control of Distributed Systems
This work addresses the problem of inefficient and opaque control in distributed systems for fields like applied sciences and engineering, offering an incremental improvement by integrating existing techniques into a novel framework.
The paper tackles the challenge of controlling high-dimensional, nonlinear partial differential equation (PDE) systems by proposing a model-based reinforcement learning framework that combines SINDy-C and autoencoders for data efficiency and interpretability, achieving improved performance on fluid flow problems like the 1D Burgers and 2D Navier-Stokes equations compared to a model-free baseline.
Effectively controlling systems governed by Partial Differential Equations (PDEs) is crucial in several fields of Applied Sciences and Engineering. These systems usually yield significant challenges to conventional control schemes due to their nonlinear dynamics, partial observability, high-dimensionality once discretized, distributed nature, and the requirement for low-latency feedback control. Reinforcement Learning (RL), particularly Deep RL (DRL), has recently emerged as a promising control paradigm for such systems, demonstrating exceptional capabilities in managing high-dimensional, nonlinear dynamics. However, DRL faces challenges including sample inefficiency, robustness issues, and an overall lack of interpretability. To address these issues, we propose a data-efficient, interpretable, and scalable Dyna-style Model-Based RL framework for PDE control, combining the Sparse Identification of Nonlinear Dynamics with Control (SINDy-C) algorithm and an autoencoder (AE) framework for the sake of dimensionality reduction of PDE states and actions. This novel approach enables fast rollouts, reducing the need for extensive environment interactions, and provides an interpretable latent space representation of the PDE forward dynamics. We validate our method on two PDE problems describing fluid flows - namely, the 1D Burgers equation and 2D Navier-Stokes equations - comparing it against a model-free baseline, and carrying out an extensive analysis of the learned dynamics.