Predict-and-recompute conjugate gradient variants
This work addresses communication bottlenecks in parallel CG solvers for scientific computing, offering a practical drop-in replacement with minimal convergence loss.
The paper introduces predict-and-recompute conjugate gradient variants that overlap global synchronizations and matrix-vector products to reduce runtime per iteration on parallel architectures, achieving convergence nearly as good as standard CG on various test problems without additional input parameters.
The standard implementation of the conjugate gradient algorithm suffers from communication bottlenecks on parallel architectures, due primarily to the two global reductions required every iteration. In this paper, we study conjugate gradient variants which decrease the runtime per iteration by overlapping global synchronizations, and in the case of pipelined variants, matrix-vector products. Through the use of a predict-and-recompute scheme, whereby recursively-updated quantities are first used as a predictor for their true values and then recomputed exactly at a later point in the iteration, these variants are observed to have convergence behavior nearly as good as the standard conjugate gradient implementation on a variety of test problems. We provide a rounding error analysis which provides insight into this observation. It is also verified experimentally that the variants studied do indeed reduce the runtime per iteration in practice and that they scale similarly to previously-studied communication-hiding variants. Finally, because these variants achieve good convergence without the use of any additional input parameters, they have the potential to be used in place of the standard conjugate gradient implementation in a range of applications.