Smoothing the Landscape: Causal Structure Learning via Diffusion Denoising Objectives
This work addresses scalability and stability issues in causal discovery for decision-making, representing an incremental improvement over existing methods like NOTEARS and DAG-GNN.
The paper tackles the problem of causal structure learning in high-dimensional observational data by introducing a diffusion denoising objective to smooth gradients for faster, more stable convergence, and demonstrates competitive performance on synthetic benchmarks with qualitative analyses on real-world examples.
Understanding causal dependencies in observational data is critical for informing decision-making. These relationships are often modeled as Bayesian Networks (BNs) and Directed Acyclic Graphs (DAGs). Existing methods, such as NOTEARS and DAG-GNN, often face issues with scalability and stability in high-dimensional data, especially when there is a feature-sample imbalance. Here, we show that the denoising score matching objective of diffusion models could smooth the gradients for faster, more stable convergence. We also propose an adaptive k-hop acyclicity constraint that improves runtime over existing solutions that require matrix inversion. We name this framework Denoising Diffusion Causal Discovery (DDCD). Unlike generative diffusion models, DDCD utilizes the reverse denoising process to infer a parameterized causal structure rather than to generate data. We demonstrate the competitive performance of DDCDs on synthetic benchmarking data. We also show that our methods are practically useful by conducting qualitative analyses on two real-world examples. Code is available at this url: https://github.com/haozhu233/ddcd.