Fast algorithm for overcomplete order-3 tensor decomposition
This work provides a more efficient method for tensor decomposition, a key task in machine learning and data analysis, though it is incremental as it builds on existing spectral and fast algorithm approaches.
The authors tackled the problem of decomposing overcomplete third-order tensors by developing the first fast spectral algorithm that handles ranks up to O(d^{3/2}/polylog(d)) and recovers components in O(d^{6.05}) time, improving upon prior fast algorithms limited to O(d^{4/3}/polylog(d)) ranks.
We develop the first fast spectral algorithm to decompose a random third-order tensor over $\mathbb{R}^d$ of rank up to $O(d^{3/2}/\text{polylog}(d))$. Our algorithm only involves simple linear algebra operations and can recover all components in time $O(d^{6.05})$ under the current matrix multiplication time. Prior to this work, comparable guarantees could only be achieved via sum-of-squares [Ma, Shi, Steurer 2016]. In contrast, fast algorithms [Hopkins, Schramm, Shi, Steurer 2016] could only decompose tensors of rank at most $O(d^{4/3}/\text{polylog}(d))$. Our algorithmic result rests on two key ingredients. A clean lifting of the third-order tensor to a sixth-order tensor, which can be expressed in the language of tensor networks. A careful decomposition of the tensor network into a sequence of rectangular matrix multiplications, which allows us to have a fast implementation of the algorithm.