Compact Factorization of Matrices Using Generalized Round-Rank
This addresses the problem of compactly representing large, noisy matrices for machine learning applications, offering a novel alternative to traditional linear factorization methods.
The paper tackles the problem of compact matrix factorization by introducing generalized round-rank (GRR), a new notion based on non-linear link functions and ordinal rounding. The result shows that GRR-based factorization achieves significantly higher accuracy than linear factorization while using lower-rank representations and converging faster on real-world datasets.
Matrix factorization is a well-studied task in machine learning for compactly representing large, noisy data. In our approach, instead of using the traditional concept of matrix rank, we define a new notion of link-rank based on a non-linear link function used within factorization. In particular, by applying the round function on a factorization to obtain ordinal-valued matrices, we introduce generalized round-rank (GRR). We show that not only are there many full-rank matrices that are low GRR, but further, that these matrices cannot be approximated well by low-rank linear factorization. We provide uniqueness conditions of this formulation and provide gradient descent-based algorithms. Finally, we present experiments on real-world datasets to demonstrate that the GRR-based factorization is significantly more accurate than linear factorization, while converging faster and using lower rank representations.