Local Loss Optimization in Operator Models: A New Insight into Spectral Learning
This work provides incremental improvements to spectral learning methods for researchers in machine learning and statistics.
The paper tackles the problem of learning latent variable models via spectral methods by introducing a local loss optimization approach, showing that a regularized convex relaxation improves the trade-off between accuracy and model complexity in practice.
This paper re-visits the spectral method for learning latent variable models defined in terms of observable operators. We give a new perspective on the method, showing that operators can be recovered by minimizing a loss defined on a finite subset of the domain. A non-convex optimization similar to the spectral method is derived. We also propose a regularized convex relaxation of this optimization. We show that in practice the availabilty of a continuous regularization parameter (in contrast with the discrete number of states in the original method) allows a better trade-off between accuracy and model complexity. We also prove that in general, a randomized strategy for choosing the local loss will succeed with high probability.