Mechanistic Independence: A Principle for Identifiable Disentangled Representations
This addresses a foundational issue in machine learning for researchers and practitioners by providing a unified approach to disentanglement without relying on statistical assumptions, though it appears incremental as it builds on existing disentanglement concepts.
The paper tackles the problem of identifiability in disentangled representations by introducing a framework based on mechanistic independence, which characterizes latent factors by their action on observed variables rather than latent distributions, and shows that this yields identifiability of latent subspaces under nonlinear, non-invertible mixing.
Disentangled representations seek to recover latent factors of variation underlying observed data, yet their identifiability is still not fully understood. We introduce a unified framework in which disentanglement is achieved through mechanistic independence, which characterizes latent factors by how they act on observed variables rather than by their latent distribution. This perspective is invariant to changes of the latent density, even when such changes induce statistical dependencies among factors. Within this framework, we propose several related independence criteria -- ranging from support-based and sparsity-based to higher-order conditions -- and show that each yields identifiability of latent subspaces, even under nonlinear, non-invertible mixing. We further establish a hierarchy among these criteria and provide a graph-theoretic characterization of latent subspaces as connected components. Together, these results clarify the conditions under which disentangled representations can be identified without relying on statistical assumptions.