Hierarchical Methods of Moments
This work addresses robustness issues in spectral methods for practitioners in machine learning, but it is incremental as it builds on existing tensor decomposition techniques.
The paper tackles the problem of spectral methods of moments lacking robustness to model misspecification in latent variable models, and presents a hierarchical approach that replaces tensor decomposition with approximate joint diagonalization, resulting in improved speed and model quality in topic modeling experiments.
Spectral methods of moments provide a powerful tool for learning the parameters of latent variable models. Despite their theoretical appeal, the applicability of these methods to real data is still limited due to a lack of robustness to model misspecification. In this paper we present a hierarchical approach to methods of moments to circumvent such limitations. Our method is based on replacing the tensor decomposition step used in previous algorithms with approximate joint diagonalization. Experiments on topic modeling show that our method outperforms previous tensor decomposition methods in terms of speed and model quality.