Polynomial Tensor Sketch for Element-wise Function of Low-Rank Matrix
This work addresses a computational bottleneck in handling non-linear functions on large matrices, offering an incremental improvement for machine learning applications.
The paper tackles the problem of approximating element-wise functions of low-rank matrices efficiently, proposing a sketching-based algorithm that reduces complexity by avoiding explicit access to all matrix entries, with demonstrated applicability in machine learning tasks.
This paper studies how to sketch element-wise functions of low-rank matrices. Formally, given low-rank matrix A = [Aij] and scalar non-linear function f, we aim for finding an approximated low-rank representation of the (possibly high-rank) matrix [f(Aij)]. To this end, we propose an efficient sketching-based algorithm whose complexity is significantly lower than the number of entries of A, i.e., it runs without accessing all entries of [f(Aij)] explicitly. The main idea underlying our method is to combine a polynomial approximation of f with the existing tensor sketch scheme for approximating monomials of entries of A. To balance the errors of the two approximation components in an optimal manner, we propose a novel regression formula to find polynomial coefficients given A and f. In particular, we utilize a coreset-based regression with a rigorous approximation guarantee. Finally, we demonstrate the applicability and superiority of the proposed scheme under various machine learning tasks.