Probabilistic Interpretation of Linear Solvers
This provides a foundation for new nonlinear optimization methods, addressing error estimation in linear solvers for computational mathematics and optimization domains.
The paper tackles the problem of solving unconstrained linear equations by proposing a probabilistic framework that replaces point estimates with Gaussian posteriors for error estimation, resulting in uncertainty-calibrated methods with minimal cost overhead over conjugate gradients.
This manuscript proposes a probabilistic framework for algorithms that iteratively solve unconstrained linear problems $Bx = b$ with positive definite $B$ for $x$. The goal is to replace the point estimates returned by existing methods with a Gaussian posterior belief over the elements of the inverse of $B$, which can be used to estimate errors. Recent probabilistic interpretations of the secant family of quasi-Newton optimization algorithms are extended. Combined with properties of the conjugate gradient algorithm, this leads to uncertainty-calibrated methods with very limited cost overhead over conjugate gradients, a self-contained novel interpretation of the quasi-Newton and conjugate gradient algorithms, and a foundation for new nonlinear optimization methods.