Probabilistic selection of inducing points in sparse Gaussian processes
This work addresses a bottleneck in sparse Gaussian processes for practitioners by introducing a Bayesian approach to inducing point selection, though it is incremental as it builds on existing variational methods.
The authors tackled the problem of selecting inducing points in sparse Gaussian processes by placing a point process prior on them and approximating the posterior with stochastic variational inference, enabling the model to learn the number and selection of points, resulting in fewer points being used as they become less informative.
Sparse Gaussian processes and various extensions thereof are enabled through inducing points, that simultaneously bottleneck the predictive capacity and act as the main contributor towards model complexity. However, the number of inducing points is generally not associated with uncertainty which prevents us from applying the apparatus of Bayesian reasoning for identifying an appropriate trade-off. In this work we place a point process prior on the inducing points and approximate the associated posterior through stochastic variational inference. By letting the prior encourage a moderate number of inducing points, we enable the model to learn which and how many points to utilise. We experimentally show that fewer inducing points are preferred by the model as the points become less informative, and further demonstrate how the method can be employed in deep Gaussian processes and latent variable modelling.