Spectral Experts for Estimating Mixtures of Linear Regressions
This addresses the challenge of local optima in discriminative latent-variable models for researchers in machine learning, though it is incremental as it focuses on a specific model instance.
The paper tackles the problem of learning mixtures of linear regressions by developing a new estimator that avoids local optima issues of EM or gradient-based methods, achieving provable consistency and computational efficiency with demonstrated empirical improvements over EM.
Discriminative latent-variable models are typically learned using EM or gradient-based optimization, which suffer from local optima. In this paper, we develop a new computationally efficient and provably consistent estimator for a mixture of linear regressions, a simple instance of a discriminative latent-variable model. Our approach relies on a low-rank linear regression to recover a symmetric tensor, which can be factorized into the parameters using a tensor power method. We prove rates of convergence for our estimator and provide an empirical evaluation illustrating its strengths relative to local optimization (EM).