Linear Causal Representation Learning by Topological Ordering, Pruning, and Disentanglement
This work addresses the challenge of causal interpretability in AI for researchers and practitioners, though it is incremental as it builds on existing linear CRL methods with relaxed assumptions.
The paper tackles the problem of linear causal representation learning under weaker assumptions than existing methods, proposing a novel algorithm that recovers latent causal features up to an equivalence class and demonstrates superiority in synthetic experiments and LLM interpretability analysis.
Causal representation learning (CRL) has garnered increasing interests from the causal inference and artificial intelligence community, due to its capability of disentangling potentially complex data-generating mechanism into causally interpretable latent features, by leveraging the heterogeneity of modern datasets. In this paper, we further contribute to the CRL literature, by focusing on the stylized linear structural causal model over the latent features and assuming a linear mixing function that maps latent features to the observed data or measurements. Existing linear CRL methods often rely on stringent assumptions, such as accessibility to single-node interventional data or restrictive distributional constraints on latent features and exogenous measurement noise. However, these prerequisites can be challenging to satisfy in certain scenarios. In this work, we propose a novel linear CRL algorithm that, unlike most existing linear CRL methods, operates under weaker assumptions about environment heterogeneity and data-generating distributions while still recovering latent causal features up to an equivalence class. We further validate our new algorithm via synthetic experiments and an interpretability analysis of large language models (LLMs), demonstrating both its superiority over competing methods in finite samples and its potential in integrating causality into AI.