Identifiability of latent-variable and structural-equation models: from linear to nonlinear
This work tackles the fundamental problem of parameter identifiability in statistical models, which is crucial for researchers in multivariate statistics and machine learning, but it is primarily a review of existing theory rather than presenting new results.
The paper reviews identifiability theory for latent-variable and structural-equation models, addressing the problem of unidentifiable parameters in linear Gaussian models and extending to nonlinear cases, showing that non-Gaussianity or auxiliary variables like time series can ensure identifiability.
An old problem in multivariate statistics is that linear Gaussian models are often unidentifiable, i.e. some parameters cannot be uniquely estimated. In factor (component) analysis, an orthogonal rotation of the factors is unidentifiable, while in linear regression, the direction of effect cannot be identified. For such linear models, non-Gaussianity of the (latent) variables has been shown to provide identifiability. In the case of factor analysis, this leads to independent component analysis, while in the case of the direction of effect, non-Gaussian versions of structural equation modelling solve the problem. More recently, we have shown how even general nonparametric nonlinear versions of such models can be estimated. Non-Gaussianity is not enough in this case, but assuming we have time series, or that the distributions are suitably modulated by some observed auxiliary variables, the models are identifiable. This paper reviews the identifiability theory for the linear and nonlinear cases, considering both factor analytic models and structural equation models.