Identifiability of Potentially Degenerate Gaussian Mixture Models With Piecewise Affine Mixing
For researchers in causal representation learning, this work extends identifiability guarantees to degenerate Gaussian mixtures, addressing a challenging setting where standard density-based methods fail.
This paper addresses identifiability in causal representation learning for latent variables following a potentially degenerate Gaussian mixture model with a piecewise affine mixing function. The authors prove identifiability up to permutation and scaling using sparsity regularization and propose a two-stage estimation method that recovers ground-truth latent variables effectively on synthetic and image data.
Causal representation learning (CRL) aims to identify the underlying latent variables from high-dimensional observations, even when variables are dependent with each other. We study this problem for latent variables that follow a potentially degenerate Gaussian mixture distribution and that are only observed through the transformation via a piecewise affine mixing function. We provide a series of progressively stronger identifiability results for this challenging setting in which the probability density functions are ill-defined because of the potential degeneracy. For identifiability up to permutation and scaling, we leverage a sparsity regularization on the learned representation. Based on our theoretical results, we propose a two-stage method to estimate the latent variables by enforcing sparsity and Gaussianity in the learned representations. Experiments on synthetic and image data highlight our method's effectiveness in recovering the ground-truth latent variables.