Improving Iterative Gaussian Processes via Warm Starting Sequential Posteriors
This work addresses incremental data addition tasks in scalable GP inference, representing an incremental improvement.
The paper tackles the challenge of improving scalability in iterative Gaussian processes for sequential decision-making by proposing a method that accelerates solver convergence using solutions from smaller systems, achieving speed-ups in solving to tolerance and enhanced Bayesian optimization performance under fixed compute budgets.
Scalable Gaussian process (GP) inference is essential for sequential decision-making tasks, yet improving GP scalability remains a challenging problem with many open avenues of research. This paper focuses on iterative GPs, where iterative linear solvers, such as conjugate gradients, stochastic gradient descent or alternative projections, are used to approximate the GP posterior. We propose a new method which improves solver convergence of a large linear system by leveraging the known solution to a smaller system contained within. This is significant for tasks with incremental data additions, and we show that our technique achieves speed-ups when solving to tolerance, as well as improved Bayesian optimisation performance under a fixed compute budget.