Gaussian Process Regression and Classification under Mathematical Constraints with Learning Guarantees
This work addresses the need for flexible and theoretically sound constrained models in machine learning, particularly for applications requiring mathematical guarantees, but it appears incremental as it builds on existing Gaussian process frameworks.
The paper tackles the problem of incorporating mathematical constraints like non-negativity and monotonicity into Gaussian process models, resulting in a constrained Gaussian process (CGP) that maintains closed-form probability density functions and posterior distributions for regression and classification. It shows that CGP inherits optimal theoretical properties such as rates of posterior contraction from Gaussian processes.
We introduce constrained Gaussian process (CGP), a Gaussian process model for random functions that allows easy placement of mathematical constrains (e.g., non-negativity, monotonicity, etc) on its sample functions. CGP comes with closed-form probability density function (PDF), and has the attractive feature that its posterior distributions for regression and classification are again CGPs with closed-form expressions. Furthermore, we show that CGP inherents the optimal theoretical properties of the Gaussian process, e.g. rates of posterior contraction, due to the fact that CGP is an Gaussian process with a more efficient model space.