A New Sampling Technique for Tensors
This addresses computational bottlenecks in tensor-based methods for machine learning practitioners, though it appears incremental as it builds on existing algorithms.
The paper tackles the problem of sampling third-order tensors to speed up tensor algorithms in machine learning, proposing a biased random sampling technique that selects O(n^1.5/ε^2) elements out of n^3 to achieve goals like sparsification, completion, and factorization.
In this paper we propose new techniques to sample arbitrary third-order tensors, with an objective of speeding up tensor algorithms that have recently gained popularity in machine learning. Our main contribution is a new way to select, in a biased random way, only $O(n^{1.5}/ε^2)$ of the possible $n^3$ elements while still achieving each of the three goals: \\ {\em (a) tensor sparsification}: for a tensor that has to be formed from arbitrary samples, compute very few elements to get a good spectral approximation, and for arbitrary orthogonal tensors {\em (b) tensor completion:} recover an exactly low-rank tensor from a small number of samples via alternating least squares, or {\em (c) tensor factorization:} approximating factors of a low-rank tensor corrupted by noise. \\ Our sampling can be used along with existing tensor-based algorithms to speed them up, removing the computational bottleneck in these methods.