Efficient Learning of Mixed Membership Models
This work provides an incremental improvement for researchers and practitioners dealing with high-dimensional data in mixed membership models, such as topic modeling or network analysis.
The paper tackles the computational inefficiency of learning mixed membership models with many variables by introducing an algorithm that reduces complexity from O(p^3) to factorizing O(p/k) sub-tensors of size O(k^3), and addresses negative entry issues in estimators, achieving competitive results on simulated and real data.
We present an efficient algorithm for learning mixed membership models when the number of variables $p$ is much larger than the number of hidden components $k$. This algorithm reduces the computational complexity of state-of-the-art tensor methods, which require decomposing an $O\left(p^3\right)$ tensor, to factorizing $O\left(p/k\right)$ sub-tensors each of size $O\left(k^3\right)$. In addition, we address the issue of negative entries in the empirical method of moments based estimators. We provide sufficient conditions under which our approach has provable guarantees. Our approach obtains competitive empirical results on both simulated and real data.