A Non-monotone Alternating Updating Method for A Class of Matrix Factorization Problems
This work addresses matrix factorization problems in machine learning and imaging sciences, but it appears incremental as it builds on existing methods with specific improvements.
The authors tackled a general matrix factorization problem that is nonconvex, nonsmooth, and non-Lipschitz by developing a non-monotone alternating updating method, and numerical experiments on real datasets showed it outperformed existing methods for non-negative matrix factorization and matrix completion.
In this paper we consider a general matrix factorization model which covers a large class of existing models with many applications in areas such as machine learning and imaging sciences. To solve this possibly nonconvex, nonsmooth and non-Lipschitz problem, we develop a non-monotone alternating updating method based on a potential function. Our method essentially updates two blocks of variables in turn by inexactly minimizing this potential function, and updates another auxiliary block of variables using an explicit formula. The special structure of our potential function allows us to take advantage of efficient computational strategies for non-negative matrix factorization to perform the alternating minimization over the two blocks of variables. A suitable line search criterion is also incorporated to improve the numerical performance. Under some mild conditions, we show that the line search criterion is well defined, and establish that the sequence generated is bounded and any cluster point of the sequence is a stationary point. Finally, we conduct some numerical experiments using real datasets to compare our method with some existing efficient methods for non-negative matrix factorization and matrix completion. The numerical results show that our method can outperform these methods for these specific applications.