Mixtures of Gaussian process experts based on kernel stick-breaking processes
This work addresses scalability and predictive performance issues in Gaussian processes for machine learning applications, representing an incremental improvement over existing models.
The authors tackled the limitations of standard Gaussian processes in scalability and predictive performance by proposing a new mixture model based on kernel stick-breaking processes, which improved predictive performance as demonstrated in experiments on six datasets.
Mixtures of Gaussian process experts is a class of models that can simultaneously address two of the key limitations inherent in standard Gaussian processes: scalability and predictive performance. In particular, models that use Dirichlet processes as gating functions permit straightforward interpretation and automatic selection of the number of experts in a mixture. While the existing models are intuitive and capable of capturing non-stationarity, multi-modality and heteroskedasticity, the simplicity of their gating functions may limit the predictive performance when applied to complex data-generating processes. Capitalising on the recent advancement in the dependent Dirichlet processes literature, we propose a new mixture model of Gaussian process experts based on kernel stick-breaking processes. Our model maintains the intuitive appeal yet improve the performance of the existing models. To make it practical, we design a sampler for posterior computation based on the slice sampling. The model behaviour and improved predictive performance are demonstrated in experiments using six datasets.