On Causally Disentangled Representations
This work addresses the need for better evaluation in disentangled representation learning, which is crucial for applications like fairness and interpretability in AI, but it is incremental as it focuses on metrics rather than a new method.
The authors tackled the problem of evaluating disentangled representations from a causal perspective, proposing new metrics and a dataset to assess how well these representations capture causal generative processes, and found that existing state-of-the-art methods often fail to meet causal disentanglement criteria.
Representation learners that disentangle factors of variation have already proven to be important in addressing various real world concerns such as fairness and interpretability. Initially consisting of unsupervised models with independence assumptions, more recently, weak supervision and correlated features have been explored, but without a causal view of the generative process. In contrast, we work under the regime of a causal generative process where generative factors are either independent or can be potentially confounded by a set of observed or unobserved confounders. We present an analysis of disentangled representations through the notion of disentangled causal process. We motivate the need for new metrics and datasets to study causal disentanglement and propose two evaluation metrics and a dataset. We show that our metrics capture the desiderata of disentangled causal process. Finally, we perform an empirical study on state of the art disentangled representation learners using our metrics and dataset to evaluate them from causal perspective.