Hierarchical-Hyperplane Kernels for Actively Learning Gaussian Process Models of Nonstationary Systems
This work addresses the challenge of efficiently modeling nonlinear and nonstationary systems in machine learning, such as computer simulations and physical machines, with an incremental improvement in kernel flexibility for active learning.
The paper tackled the problem of learning surrogate models for complex, nonstationary systems with limited data by introducing a new kernel family for Gaussian processes that enables learnable, flexible input partitioning. The result is a method that achieves excellent performance on various active learning tasks, as demonstrated empirically.
Learning precise surrogate models of complex computer simulations and physical machines often require long-lasting or expensive experiments. Furthermore, the modeled physical dependencies exhibit nonlinear and nonstationary behavior. Machine learning methods that are used to produce the surrogate model should therefore address these problems by providing a scheme to keep the number of queries small, e.g. by using active learning and be able to capture the nonlinear and nonstationary properties of the system. One way of modeling the nonstationarity is to induce input-partitioning, a principle that has proven to be advantageous in active learning for Gaussian processes. However, these methods either assume a known partitioning, need to introduce complex sampling schemes or rely on very simple geometries. In this work, we present a simple, yet powerful kernel family that incorporates a partitioning that: i) is learnable via gradient-based methods, ii) uses a geometry that is more flexible than previous ones, while still being applicable in the low data regime. Thus, it provides a good prior for active learning procedures. We empirically demonstrate excellent performance on various active learning tasks.