Invariant Structure Learning for Better Generalization and Causal Explainability
This work addresses the challenge of improving generalization and causal explainability in machine learning, particularly for datasets with distribution shifts, but it appears incremental as it builds on existing causal structure learning methods.
The paper tackles the problem of causal structure discovery by proposing Invariant Structure Learning (ISL), a framework that uses generalization as an indication to learn invariant structures across different data environments, and it demonstrates accurate structure discovery and superior generalization on datasets with distribution shifts.
Learning the causal structure behind data is invaluable for improving generalization and obtaining high-quality explanations. We propose a novel framework, Invariant Structure Learning (ISL), that is designed to improve causal structure discovery by utilizing generalization as an indication. ISL splits the data into different environments, and learns a structure that is invariant to the target across different environments by imposing a consistency constraint. An aggregation mechanism then selects the optimal classifier based on a graph structure that reflects the causal mechanisms in the data more accurately compared to the structures learnt from individual environments. Furthermore, we extend ISL to a self-supervised learning setting where accurate causal structure discovery does not rely on any labels. This self-supervised ISL utilizes invariant causality proposals by iteratively setting different nodes as targets. On synthetic and real-world datasets, we demonstrate that ISL accurately discovers the causal structure, outperforms alternative methods, and yields superior generalization for datasets with significant distribution shifts.