Nonparametric Estimation of Multi-View Latent Variable Models
This work addresses the limitation of spectral methods to discrete or Gaussian distributions, enabling nonparametric estimation for researchers in machine learning and statistics, though it is incremental in extending spectral techniques to broader settings.
The paper tackles the problem of learning multi-view latent variable models with nonparametric mixture components, proposing a kernel method that embeds distributions into a reproducing kernel Hilbert space and uses a robust tensor power method. The result is a method with quadratic sample complexity in latent components and favorable performance compared to EM and other spectral algorithms in experiments.
Spectral methods have greatly advanced the estimation of latent variable models, generating a sequence of novel and efficient algorithms with strong theoretical guarantees. However, current spectral algorithms are largely restricted to mixtures of discrete or Gaussian distributions. In this paper, we propose a kernel method for learning multi-view latent variable models, allowing each mixture component to be nonparametric. The key idea of the method is to embed the joint distribution of a multi-view latent variable into a reproducing kernel Hilbert space, and then the latent parameters are recovered using a robust tensor power method. We establish that the sample complexity for the proposed method is quadratic in the number of latent components and is a low order polynomial in the other relevant parameters. Thus, our non-parametric tensor approach to learning latent variable models enjoys good sample and computational efficiencies. Moreover, the non-parametric tensor power method compares favorably to EM algorithm and other existing spectral algorithms in our experiments.