Nonparametric Factor Analysis and Beyond
This work addresses a foundational challenge in machine learning by extending identifiability results to noisy, nonparametric settings, potentially impacting researchers and practitioners dealing with latent variables in real-world scenarios.
The paper tackles the problem of identifying latent variables in unsupervised representation learning under general noise conditions, proving identifiability up to certain indeterminacies and developing estimation methods validated on synthetic and real-world data, with an example showing more insightful GDP growth estimates than official reports.
Nearly all identifiability results in unsupervised representation learning inspired by, e.g., independent component analysis, factor analysis, and causal representation learning, rely on assumptions of additive independent noise or noiseless regimes. In contrast, we study the more general case where noise can take arbitrary forms, depend on latent variables, and be non-invertibly entangled within a nonlinear function. We propose a general framework for identifying latent variables in the nonparametric noisy settings. We first show that, under suitable conditions, the generative model is identifiable up to certain submanifold indeterminacies even in the presence of non-negligible noise. Furthermore, under the structural or distributional variability conditions, we prove that latent variables of the general nonlinear models are identifiable up to trivial indeterminacies. Based on the proposed theoretical framework, we have also developed corresponding estimation methods and validated them in various synthetic and real-world settings. Interestingly, our estimate of the true GDP growth from alternative measurements suggests more insightful information on the economies than official reports. We expect our framework to provide new insight into how both researchers and practitioners deal with latent variables in real-world scenarios.