Probabilistic Linear Solvers for Machine Learning
This addresses the need for scalable and uncertainty-aware linear solvers in machine learning, though it appears incremental by unifying earlier work.
The paper tackles the challenge of solving linear systems in machine learning by proposing a class of probabilistic linear solvers that infer the matrix, its inverse, and solution from matrix-vector products, recovering conjugate gradients under certain conditions and showing potential through experimental demonstrations.
Linear systems are the bedrock of virtually all numerical computation. Machine learning poses specific challenges for the solution of such systems due to their scale, characteristic structure, stochasticity and the central role of uncertainty in the field. Unifying earlier work we propose a class of probabilistic linear solvers which jointly infer the matrix, its inverse and the solution from matrix-vector product observations. This class emerges from a fundamental set of desiderata which constrains the space of possible algorithms and recovers the method of conjugate gradients under certain conditions. We demonstrate how to incorporate prior spectral information in order to calibrate uncertainty and experimentally showcase the potential of such solvers for machine learning.