Convergence of Alternating Least Squares Optimisation for Rank-One Approximation to High Order Tensors
For researchers in tensor approximation, this provides a theoretical convergence analysis of a widely used algorithm, though it is an incremental theoretical contribution.
The paper analyzes the convergence of the alternating least squares (ALS) algorithm for rank-one tensor approximation, proving that it can converge sublinearly, Q-linearly, or Q-superlinearly, with examples provided.
The approximation of tensors has important applications in various disciplines, but it remains an extremely challenging task. It is well known that tensors of higher order can fail to have best low-rank approximations, but with an important exception that best rank-one approximations always exists. The most popular approach to low-rank approximation is the alternating least squares (ALS) method. The convergence of the alternating least squares algorithm for the rank-one approximation problem is analysed in this paper. In our analysis we are focusing on the global convergence and the rate of convergence of the ALS algorithm. It is shown that the ALS method can converge sublinearly, Q-linearly, and even Q-superlinearly. Our theoretical results are illustrated on explicit examples.