Residual Component Analysis: Generalising PCA for more flexible inference in linear-Gaussian models
This work addresses the need for more flexible inference in linear-Gaussian models for researchers in fields like bioinformatics and computer vision, though it is incremental as it builds upon existing PCA methods.
The paper tackles the problem of data variance partially explained by other factors, such as sparse dependencies or temporal correlations, by introducing Residual Component Analysis (RCA) as a generalization of PCA for linear-Gaussian models, which decomposes residual variance through a generalized eigenvalue problem and is demonstrated on protein-signaling networks, gene expression time-series, and motion capture data.
Probabilistic principal component analysis (PPCA) seeks a low dimensional representation of a data set in the presence of independent spherical Gaussian noise. The maximum likelihood solution for the model is an eigenvalue problem on the sample covariance matrix. In this paper we consider the situation where the data variance is already partially explained by other actors, for example sparse conditional dependencies between the covariates, or temporal correlations leaving some residual variance. We decompose the residual variance into its components through a generalised eigenvalue problem, which we call residual component analysis (RCA). We explore a range of new algorithms that arise from the framework, including one that factorises the covariance of a Gaussian density into a low-rank and a sparse-inverse component. We illustrate the ideas on the recovery of a protein-signaling network, a gene expression time-series data set and the recovery of the human skeleton from motion capture 3-D cloud data.