C-Disentanglement: Discovering Causally-Independent Generative Factors under an Inductive Bias of Confounder
This work addresses the challenge of causal disentanglement in representation learning for improved controllability and generalization, though it appears incremental by building on existing concepts with a new framework.
The paper tackles the problem of discovering causally-independent generative factors in representation learning by modeling confounders, which are often ignored in existing work, and introduces the C-Disentanglement framework with inductive bias from domain expertise to achieve this. The method shows competitive results on synthetic and real-world datasets compared to state-of-the-art baselines in obtaining causally disentangled features and downstream tasks under domain shifts.
Representation learning assumes that real-world data is generated by a few semantically meaningful generative factors (i.e., sources of variation) and aims to discover them in the latent space. These factors are expected to be causally disentangled, meaning that distinct factors are encoded into separate latent variables, and changes in one factor will not affect the values of the others. Compared to statistical independence, causal disentanglement allows more controllable data generation, improved robustness, and better generalization. However, most existing work assumes unconfoundedness in the discovery process, that there are no common causes to the generative factors and thus obtain only statistical independence. In this paper, we recognize the importance of modeling confounders in discovering causal generative factors. Unfortunately, such factors are not identifiable without proper inductive bias. We fill the gap by introducing a framework entitled Confounded-Disentanglement (C-Disentanglement), the first framework that explicitly introduces the inductive bias of confounder via labels from domain expertise. In addition, we accordingly propose an approach to sufficiently identify the causally disentangled factors under any inductive bias of the confounder. We conduct extensive experiments on both synthetic and real-world datasets. Our method demonstrates competitive results compared to various SOTA baselines in obtaining causally disentangled features and downstream tasks under domain shifts.