Learning Sparsity and Randomness for Data-driven Low Rank Approximation
This work provides incremental improvements for researchers and practitioners in machine learning and data analysis who rely on efficient low rank approximation algorithms.
The paper tackles the problem of improving learning-based low rank approximation by addressing limitations in learning non-zero positions and out-of-distribution performance, introducing methods that learn sparsity patterns and add randomness to sketch matrices, resulting in reduced test errors without significant complexity increase.
Learning-based low rank approximation algorithms can significantly improve the performance of randomized low rank approximation with sketch matrix. With the learned value and fixed non-zero positions for sketch matrices from learning-based algorithms, these matrices can reduce the test error of low rank approximation significantly. However, there is still no good method to learn non-zero positions as well as overcome the out-of-distribution performance loss. In this work, we introduce two new methods Learning Sparsity and Learning Randomness which try to learn a better sparsity patterns and add randomness to the value of sketch matrix. These two methods can be applied with any learning-based algorithms which use sketch matrix directly. Our experiments show that these two methods can improve the performance of previous learning-based algorithm for both test error and out-of-distribution test error without adding too much complexity.