A Unified Framework for Stochastic Matrix Factorization via Variance Reduction
This work addresses efficiency issues in matrix factorization for machine learning practitioners, but it is incremental as it builds on existing stochastic methods.
The authors tackled the problem of slow convergence in stochastic matrix factorization by proposing a unified variance reduction framework, which demonstrated faster convergence and more accurate dictionaries compared to state-of-the-art methods in experiments.
We propose a unified framework to speed up the existing stochastic matrix factorization (SMF) algorithms via variance reduction. Our framework is general and it subsumes several well-known SMF formulations in the literature. We perform a non-asymptotic convergence analysis of our framework and derive computational and sample complexities for our algorithm to converge to an $ε$-stationary point in expectation. In addition, extensive experiments for a wide class of SMF formulations demonstrate that our framework consistently yields faster convergence and a more accurate output dictionary vis-à-vis state-of-the-art frameworks.