Pencil-based algorithms for tensor rank decomposition are not stable
This work identifies a fundamental numerical instability in a widely used class of tensor decomposition algorithms, affecting practitioners in signal processing, data analysis, and scientific computing.
The paper proves that pencil-based algorithms for tensor rank decomposition are numerically unstable for an open set of tensors, due to the condition number of the decomposition being much larger for a reduced tensor. Experiments show a loss of precision of a few digits for random tensors.
We prove the existence of an open set of $n_1\times n_2 \times n_3$ tensors of rank $r$ on which a popular and efficient class of algorithms for computing tensor rank decompositions based on a reduction to a linear matrix pencil, typically followed by a generalized eigendecomposition, is arbitrarily numerically forward unstable. Our analysis shows that this problem is caused by the fact that the condition number of the tensor rank decomposition can be much larger for $n_1 \times n_2 \times 2$ tensors than for the $n_1\times n_2 \times n_3$ input tensor. Moreover, we present a lower bound for the limiting distribution of the condition number of random tensor rank decompositions of third-order tensors. The numerical experiments illustrate that for random tensor rank decompositions one should anticipate a loss of precision of a few digits.