Cross-covariance modelling via DAGs with hidden variables
This addresses difficulties in DAG models with hidden variables for researchers in statistics and machine learning, but it is incremental as it focuses on a specific class of models.
The paper tackles the problem of modeling cross-covariance in Gaussian latent variable models with hidden variables, characterizing the set of distributions for one-dimensional models and relating it to singular value decomposition, showing that useful information can be extracted despite underidentification.
DAG models with hidden variables present many difficulties that are not present when all nodes are observed. In particular, fully observed DAG models are identified and correspond to well-defined sets ofdistributions, whereas this is not true if nodes are unobserved. Inthis paper we characterize exactly the set of distributions given by a class of one-dimensional Gaussian latent variable models. These models relate two blocks of observed variables, modeling only the cross-covariance matrix. We describe the relation of this model to the singular value decomposition of the cross-covariance matrix. We show that, although the model is underidentified, useful information may be extracted. We further consider an alternative parametrization in which one latent variable is associated with each block. Our analysis leads to some novel covariance equivalence results for Gaussian hidden variable models.