Graph-attention-based Casual Discovery with Trust Region-navigated Clipping Policy Optimization
This addresses causal discovery for empirical sciences where conventional methods fail due to unoriented edges or latent assumptions, though it appears incremental as it builds on existing RL approaches.
The paper tackles the problem of causal discovery from observational data by improving reinforcement learning methods for searching directed acyclic graphs, proposing a trust region-navigated clipping policy optimization and a refined graph attention encoder called SDGAT that outperforms previous RL methods on synthetic and benchmark datasets.
In many domains of empirical sciences, discovering the causal structure within variables remains an indispensable task. Recently, to tackle with unoriented edges or latent assumptions violation suffered by conventional methods, researchers formulated a reinforcement learning (RL) procedure for causal discovery, and equipped REINFORCE algorithm to search for the best-rewarded directed acyclic graph. The two keys to the overall performance of the procedure are the robustness of RL methods and the efficient encoding of variables. However, on the one hand, REINFORCE is prone to local convergence and unstable performance during training. Neither trust region policy optimization, being computationally-expensive, nor proximal policy optimization (PPO), suffering from aggregate constraint deviation, is decent alternative for combinatory optimization problems with considerable individual subactions. We propose a trust region-navigated clipping policy optimization method for causal discovery that guarantees both better search efficiency and steadiness in policy optimization, in comparison with REINFORCE, PPO and our prioritized sampling-guided REINFORCE implementation. On the other hand, to boost the efficient encoding of variables, we propose a refined graph attention encoder called SDGAT that can grasp more feature information without priori neighbourhood information. With these improvements, the proposed method outperforms former RL method in both synthetic and benchmark datasets in terms of output results and optimization robustness.