Learning Gaussian Processes by Minimizing PAC-Bayesian Generalization Bounds
This work addresses the need for robust performance guarantees in safety-critical applications of Gaussian Processes, though it is incremental as it builds on existing PAC-Bayesian theory.
The paper tackled the problem of Gaussian Processes lacking good performance guarantees for safety-critical applications by proposing a method that learns GPs by optimizing a PAC-Bayesian bound on generalization performance, resulting in significantly better generalization guarantees than common GP approaches on regression benchmarks.
Gaussian Processes (GPs) are a generic modelling tool for supervised learning. While they have been successfully applied on large datasets, their use in safety-critical applications is hindered by the lack of good performance guarantees. To this end, we propose a method to learn GPs and their sparse approximations by directly optimizing a PAC-Bayesian bound on their generalization performance, instead of maximizing the marginal likelihood. Besides its theoretical appeal, we find in our evaluation that our learning method is robust and yields significantly better generalization guarantees than other common GP approaches on several regression benchmark datasets.