Learning to Solve Related Linear Systems
This work addresses a computational bottleneck in numerical tasks involving multiple related linear systems, offering a domain-specific improvement for applications like hyperparameter optimization.
The paper tackles the problem of solving multiple parametrized linear systems efficiently by proposing a novel probabilistic linear solver that learns from previously solved systems. The method provides an efficient posterior mean and covariance that can be used as an initial guess and preconditioner, with numerical experiments demonstrating benefits in a hyperparameter optimization problem.
Solving multiple parametrised related systems is an essential component of many numerical tasks, and learning from the already solved systems will make this process faster. In this work, we propose a novel probabilistic linear solver over the parameter space. This leverages information from the solved linear systems in a regression setting to provide an efficient posterior mean and covariance. We advocate using this as companion regression model for the preconditioned conjugate gradient method, and discuss the favourable properties of the posterior mean and covariance as the initial guess and preconditioner. We also provide several design choices for this companion solver. Numerical experiments showcase the benefits of using our novel solver in a hyperparameter optimisation problem.