Complex-to-Real Sketches for Tensor Products with Applications to the Polynomial Kernel
This provides a more efficient approximation method for tensor products, particularly beneficial for applications using polynomial kernels in machine learning.
The paper tackles the computational inefficiency of randomized sketches for tensor products, which had suboptimal embedding dimension scaling of O(3^p). By proposing a Complex-to-Real modification that uses complex random projections instead of real ones, the method achieves a lower O(2^p) factor in embedding dimension while maintaining real-valued outputs.
Randomized sketches of a tensor product of $p$ vectors follow a tradeoff between statistical efficiency and computational acceleration. Commonly used approaches avoid computing the high-dimensional tensor product explicitly, resulting in a suboptimal dependence of $\mathcal{O}(3^p)$ in the embedding dimension. We propose a simple Complex-to-Real (CtR) modification of well-known sketches that replaces real random projections by complex ones, incurring a lower $\mathcal{O}(2^p)$ factor in the embedding dimension. The output of our sketches is real-valued, which renders their downstream use straightforward. In particular, we apply our sketches to $p$-fold self-tensored inputs corresponding to the feature maps of the polynomial kernel. We show that our method achieves state-of-the-art performance in terms of accuracy and speed compared to other randomized approximations from the literature.