Restoring Chaos Using Deep Reinforcement Learning
This work addresses the challenge of preventing crises in non-linear dynamical systems, which is important for fields like physics and engineering, but it appears incremental as it applies an existing deep RL method to a specific domain problem.
The researchers tackled the problem of catastrophic bifurcations in non-linear dynamical systems, such as the Lorenz system, which can lead to undesirable non-chaotic states, and demonstrated that deep reinforcement learning can restore chaos without prior knowledge of the dynamics, achieving successful control as evidenced by the agent's discovered perturbation strategy and implemented control-law.
A catastrophic bifurcation in non-linear dynamical systems, called crisis, often leads to their convergence to an undesirable non-chaotic state after some initial chaotic transients. Preventing such behavior has proved to be quite challenging. We demonstrate that deep Reinforcement Learning (RL) is able to restore chaos in a transiently-chaotic regime of the Lorenz system of equations. Without requiring any a priori knowledge of the underlying dynamics of the governing equations, the RL agent discovers an effective perturbation strategy for sustaining the chaotic trajectory. We analyze the agent's autonomous control-decisions, and identify and implement a simple control-law that successfully restores chaos in the Lorenz system. Our results demonstrate the utility of using deep RL for controlling the occurrence of catastrophes and extreme-events in non-linear dynamical systems.