Computing Transition Pathways for the Study of Rare Events Using Deep Reinforcement Learning
This work addresses a challenging task in computational physics, chemistry, and biology for studying rare events like molecular conformational changes, but it is incremental as it applies an existing reinforcement learning method to a known bottleneck in path-finding.
The authors tackled the problem of computing transition pathways between metastable states in complex systems by formulating it as a cost minimization problem and solving it using a deep reinforcement learning method based on DDPG, achieving efficient sampling and globally optimal pathways as demonstrated on benchmark systems like the extended Mueller and Lennard-Jones systems.
Understanding the transition events between metastable states in complex systems is an important subject in the fields of computational physics, chemistry and biology. The transition pathway plays an important role in characterizing the mechanism underlying the transition, for example, in the study of conformational changes of bio-molecules. In fact, computing the transition pathway is a challenging task for complex and high-dimensional systems. In this work, we formulate the path-finding task as a cost minimization problem over a particular path space. The cost function is adapted from the Freidlin-Wentzell action functional so that it is able to deal with rough potential landscapes. The path-finding problem is then solved using a actor-critic method based on the deep deterministic policy gradient algorithm (DDPG). The method incorporates the potential force of the system in the policy for generating episodes and combines physical properties of the system with the learning process for molecular systems. The exploitation and exploration nature of reinforcement learning enables the method to efficiently sample the transition events and compute the globally optimal transition pathway. We illustrate the effectiveness of the proposed method using three benchmark systems including an extended Mueller system and the Lennard-Jones system of seven particles.