A Causal Ordering Prior for Unsupervised Representation Learning
This addresses the challenge of learning more realistic and generalizable representations in unsupervised settings, though it appears incremental as it builds on existing causal representation learning ideas.
The paper tackles the problem of unsupervised representation learning by proposing a method that incorporates a causal ordering prior to allow correlated latent variables, resulting in a fully unsupervised approach that does not require auxiliary information or interventions.
Unsupervised representation learning with variational inference relies heavily on independence assumptions over latent variables. Causal representation learning (CRL), however, argues that factors of variation in a dataset are, in fact, causally related. Allowing latent variables to be correlated, as a consequence of causal relationships, is more realistic and generalisable. So far, provably identifiable methods rely on: auxiliary information, weak labels, and interventional or even counterfactual data. Inspired by causal discovery with functional causal models, we propose a fully unsupervised representation learning method that considers a data generation process with a latent additive noise model (ANM). We encourage the latent space to follow a causal ordering via loss function based on the Hessian of the latent distribution.