Sparse Infinite Random Feature Latent Variable Modeling
This provides a method for sparse, non-linear latent variable modeling that automatically determines dimensionality, benefiting researchers in fields like biology and text analysis, though it appears incremental as it builds on existing latent variable and non-parametric techniques.
The paper tackles the problem of non-linear latent variable modeling by proposing a Bayesian non-parametric model with a sparse, infinite-dimensional latent space using an Indian buffet process prior, which automatically selects the number of latent dimensions and imposes sparsity, and demonstrates superior test set performance on synthetic, biological, and text datasets compared to previous models.
We propose a non-linear, Bayesian non-parametric latent variable model where the latent space is assumed to be sparse and infinite dimensional a priori using an Indian buffet process prior. A posteriori, the number of instantiated dimensions in the latent space is guaranteed to be finite. The purpose of placing the Indian buffet process on the latent variables is to: 1.) Automatically and probabilistically select the number of latent dimensions. 2.) Impose sparsity in the latent space, where the Indian buffet process will select which elements are exactly zero. Our proposed model allows for sparse, non-linear latent variable modeling where the number of latent dimensions is selected automatically. Inference is made tractable using the random Fourier approximation and we can easily implement posterior inference through Markov chain Monte Carlo sampling. This approach is amenable to many observation models beyond the Gaussian setting. We demonstrate the utility of our method on a variety of synthetic, biological and text datasets and show that we can obtain superior test set performance compared to previous latent variable models.