Dual Parameterization of Sparse Variational Gaussian Processes
This work improves computational efficiency for SVGP methods, which are used in non-conjugate Gaussian process inference, but it is incremental as it builds on existing SVGP techniques.
The paper tackles the computational efficiency of sparse variational Gaussian processes (SVGP) by introducing a dual parameterization that assigns dual parameters to each data example, similar to expectation propagation site parameters, resulting in faster inference and a tighter evidence lower bound for hyperparameter learning.
Sparse variational Gaussian process (SVGP) methods are a common choice for non-conjugate Gaussian process inference because of their computational benefits. In this paper, we improve their computational efficiency by using a dual parameterization where each data example is assigned dual parameters, similarly to site parameters used in expectation propagation. Our dual parameterization speeds-up inference using natural gradient descent, and provides a tighter evidence lower bound for hyperparameter learning. The approach has the same memory cost as the current SVGP methods, but it is faster and more accurate.