Accelerating Non-Conjugate Gaussian Processes By Trading Off Computation For Uncertainty
This work addresses a bottleneck in probabilistic modeling for practitioners using NCGPs on large-scale classification problems, though it appears incremental as it builds on existing approximation methods.
The paper tackles the computational expense and approximation error in non-conjugate Gaussian processes (NCGPs) for large datasets by introducing iterative methods that model this error, resulting in significant acceleration of posterior inference while trading off reduced computation for increased uncertainty.
Non-conjugate Gaussian processes (NCGPs) define a flexible probabilistic framework to model categorical, ordinal and continuous data, and are widely used in practice. However, exact inference in NCGPs is prohibitively expensive for large datasets, thus requiring approximations in practice. The approximation error adversely impacts the reliability of the model and is not accounted for in the uncertainty of the prediction. We introduce a family of iterative methods that explicitly model this error. They are uniquely suited to parallel modern computing hardware, efficiently recycle computations, and compress information to reduce both the time and memory requirements for NCGPs. As we demonstrate on large-scale classification problems, our method significantly accelerates posterior inference compared to competitive baselines by trading off reduced computation for increased uncertainty.