Deep Sigma Point Processes
This work addresses the need for well-calibrated uncertainty in scalable regression for machine learning practitioners, though it is incremental as it builds on existing Deep Gaussian Process frameworks.
The paper tackled the problem of scalable probabilistic regression by introducing Deep Sigma Point Processes (DSPPs), which avoid posterior approximations through maximum likelihood inference, resulting in predictive distributions that are significantly better calibrated than variational Deep Gaussian Processes, often by as much as a nat per datapoint.
We introduce Deep Sigma Point Processes, a class of parametric models inspired by the compositional structure of Deep Gaussian Processes (DGPs). Deep Sigma Point Processes (DSPPs) retain many of the attractive features of (variational) DGPs, including mini-batch training and predictive uncertainty that is controlled by kernel basis functions. Importantly, since DSPPs admit a simple maximum likelihood inference procedure, the resulting predictive distributions are not degraded by any posterior approximations. In an extensive empirical comparison on univariate and multivariate regression tasks we find that the resulting predictive distributions are significantly better calibrated than those obtained with other probabilistic methods for scalable regression, including variational DGPs--often by as much as a nat per datapoint.