Causal Coupled Mechanisms: A Control Method with Cooperation and Competition for Complex System
This addresses control challenges in complex systems like biological regulation, offering improved accuracy and generalization, though it appears incremental as it builds on hierarchical reinforcement learning.
The paper tackles the control of complex systems with complicated dynamics by proposing Causal Coupled Mechanisms (CCMs), a method that divides systems into modular structures and achieves robust, state-of-the-art control results on synthetic and real-world biological systems, even with unpredictable random noise.
Complex systems are ubiquitous in the real world and tend to have complicated and poorly understood dynamics. For their control issues, the challenge is to guarantee accuracy, robustness, and generalization in such bloated and troubled environments. Fortunately, a complex system can be divided into multiple modular structures that human cognition appears to exploit. Inspired by this cognition, a novel control method, Causal Coupled Mechanisms (CCMs), is proposed that explores the cooperation in division and competition in combination. Our method employs the theory of hierarchical reinforcement learning (HRL), in which 1) the high-level policy with competitive awareness divides the whole complex system into multiple functional mechanisms, and 2) the low-level policy finishes the control task of each mechanism. Specifically for cooperation, a cascade control module helps the series operation of CCMs, and a forward coupled reasoning module is used to recover the coupling information lost in the division process. On both synthetic systems and a real-world biological regulatory system, the CCM method achieves robust and state-of-the-art control results even with unpredictable random noise. Moreover, generalization results show that reusing prepared specialized CCMs helps to perform well in environments with different confounders and dynamics.