Concept Modulation Models: A Unified Framework for Identifiability and Extrapolation
This work provides a general theoretical framework that unifies and extends identifiability and extrapolation guarantees across multiple model families, offering algebraic criteria for practitioners.
The paper introduces concept modulation models (CMMs), a unified framework for analyzing identifiability and extrapolation in conditional latent variable models. It shows that feature agreement on observed attributes induces latent concept transitions expressed via attribute potentials, which also control extrapolation, recovering existing results from nonlinear ICA, causal representation learning, and perturbation modeling.
Reliable generalization in conditional latent variable models requires understanding both identifiability and extrapolation: how observed variation across attributes determines latent structure, and how that structure determines distributions at unseen attributes. However, existing identifiability and extrapolation guarantees are largely model-specific, with separate analyses in nonlinear ICA, causal representation learning, perturbation modeling, and related conditional latent variable models. We introduce concept modulation models (CMMs), an attribute-indexed class of conditional generative models with structure $A\to Λ\to C\to X$, where attributes select modulators, modulators induce latent concept laws, and concepts generate observed features. CMMs lift transition-based identifiability to conditional settings by showing that feature agreement on observed attributes induces a latent concept transition constrained by the CMM class. We express these constraints through attribute potentials, log-density ratios between attribute-conditioned concept laws, separating the generic lifting step from model-specific rigidity arguments. The same potentials control extrapolation: agreement at unseen attributes holds exactly when the transported attribute-potential identities extend to those attributes. This yields algebraic extrapolation criteria, identifies the common potential-based proof objects behind several existing identifiability and extrapolation results, and, when combined with the model-specific rigidity arguments in those works, recovers their stated conclusions.