Linear convergence of Gearhart-Koshy accelerated Kaczmarz methods for tensor linear systems
This provides a theoretical foundation for accelerating iterative solvers in numerical linear algebra, particularly for tensor systems, but is incremental as it builds on existing acceleration techniques.
The paper tackles the problem of establishing linear convergence for the generalized Gearhart-Koshy accelerated Kaczmarz method applied to tensor linear systems, proving it converges linearly to the unique least-norm solution and showing it yields a better convergence upper bound than the plain method.
The generalized Gearhart-Koshy acceleration is a recent exact affine search technique designed for the method of cyclic projections onto hyperplanes, i.e., the Kaczmarz method. However, its convergence properties, particularly the linear convergence rate, have not been thoroughly established. In this paper, we systematically establish the linear convergence of the generalized Gearhart-Koshy accelerated Kaczmarz method for tensor linear systems, proving that it converges linearly to the unique least-norm solution. Our analysis is general and applies to several popular Kaczmarz variants, including incremental, shuffle-once, and random-reshuffling schemes, and demonstrates that this acceleration approach yields a better convergence upper bound compared to the plain Kaczmarz method. We also propose an efficient Gram-Schmidt-based implementation that computes the next iterate in linear time. Building on this implementation, we establish a connection between this acceleration framework and Arnoldi-type Krylov subspace methods, further highlighting its efficiency and potential. Our theoretical results are supported by numerical experiments.