Preconditioning Kaczmarz method by sketching
This work addresses the need for faster convergence in iterative linear system solvers, but the improvement is incremental as it applies known sketching techniques to precondition the Kaczmarz method.
The authors propose a sketching-based preconditioner for the Kaczmarz method to accelerate convergence in solving overdetermined linear systems, demonstrating improved performance over the unpreconditioned method in numerical experiments on random and real data.
We propose a new method for preconditioning Kaczmarz method by sketching. Kaczmarz method is a stochastic method for solving overdetermined linear systems based on a sampling of matrix rows. The standard approach to speed up convergence of iterative methods is using preconditioner. As we show the best possible preconditioner for this method can be constructed from QR decomposition of the system matrix, but the complexity of this procedure is too high. Therefore, to reduce this complexity, we use random sketching and compare it with the Kaczmarz method without preconditioning. The developed method is applicable for different modifications of classical Kaczmarz method that were proposed recently. We provide numerical experiments to show the performance of the developed methods on solving both random and real overdetermined linear systems.