Efficient Tensor Contraction via Fast Count Sketch
This work addresses efficiency and accuracy issues in tensor-based applications for machine learning and data analysis, representing an incremental improvement over existing sketching methods.
The paper tackles the problem of low accuracy or slow speed in existing tensor sketching methods by proposing Fast Count Sketch (FCS), which applies multiple shorter hash functions to preserve spatial information and uses fast Fourier transform for acceleration, achieving superior approximation accuracy and computational efficiency in applications like tensor decomposition and compression.
Sketching uses randomized Hash functions for dimensionality reduction and acceleration. The existing sketching methods, such as count sketch (CS), tensor sketch (TS), and higher-order count sketch (HCS), either suffer from low accuracy or slow speed in some tensor based applications. In this paper, the proposed fast count sketch (FCS) applies multiple shorter Hash functions based CS to the vector form of the input tensor, which is more accurate than TS since the spatial information of the input tensor can be preserved more sufficiently. When the input tensor admits CANDECOMP/PARAFAC decomposition (CPD), FCS can accelerate CS and HCS by using fast Fourier transform, which exhibits a computational complexity asymptotically identical to TS for low-order tensors. The effectiveness of FCS is validated by CPD, tensor regression network compression, and Kronecker product compression. Experimental results show its superior performance in terms of approximation accuracy and computational efficiency.