Additive Gaussian Processes Revisited
This work addresses the interpretability and computational efficiency of Gaussian process models for researchers and practitioners in machine learning, representing an incremental improvement.
The paper tackled the problem of additive Gaussian process models requiring high-dimensional interaction terms by proposing the orthogonal additive kernel (OAK) with an orthogonality constraint, achieving similar or better predictive performance with only a small number of low-dimensional terms while retaining interpretability.
Gaussian Process (GP) models are a class of flexible non-parametric models that have rich representational power. By using a Gaussian process with additive structure, complex responses can be modelled whilst retaining interpretability. Previous work showed that additive Gaussian process models require high-dimensional interaction terms. We propose the orthogonal additive kernel (OAK), which imposes an orthogonality constraint on the additive functions, enabling an identifiable, low-dimensional representation of the functional relationship. We connect the OAK kernel to functional ANOVA decomposition, and show improved convergence rates for sparse computation methods. With only a small number of additive low-dimensional terms, we demonstrate the OAK model achieves similar or better predictive performance compared to black-box models, while retaining interpretability.