Weighted Low Rank Matrix Approximation and Acceleration
This work addresses a generalization of low-rank matrix approximation for applications such as recommender systems and statistical modeling, but it appears incremental as it builds on existing LRMA and LRMC methods.
The paper tackles the problem of element-wise weighted low-rank matrix approximation, which generalizes low-rank matrix completion and is useful for applications like GLM optimization with exponential families. The authors propose an algorithm for solving the weighted problem, along with acceleration techniques and a non-SVD modification for high-dimensional data, and compare performance on simulation and real-data examples.
Low-rank matrix approximation is one of the central concepts in machine learning, with applications in dimension reduction, de-noising, multivariate statistical methodology, and many more. A recent extension to LRMA is called low-rank matrix completion (LRMC). It solves the LRMA problem when some observations are missing and is especially useful for recommender systems. In this paper, we consider an element-wise weighted generalization of LRMA. The resulting weighted low-rank matrix approximation technique therefore covers LRMC as a special case with binary weights. WLRMA has many applications. For example, it is an essential component of GLM optimization algorithms, where an exponential family is used to model the entries of a matrix, and the matrix of natural parameters admits a low-rank structure. We propose an algorithm for solving the weighted problem, as well as two acceleration techniques. Further, we develop a non-SVD modification of the proposed algorithm that is able to handle extremely high-dimensional data. We compare the performance of all the methods on a small simulation example as well as a real-data application.