Amortized Inference for Gaussian Process Hyperparameters of Structured Kernels
This addresses the problem of slow training times in Gaussian process applications like online learning and Bayesian optimization, offering a more flexible and efficient solution, though it is incremental in extending amortization to kernel families.
The paper tackles the computational bottleneck of learning kernel parameters for Gaussian processes by proposing an amortized inference method that works over entire kernel-structure families, rather than fixed kernels, drastically reducing inference time while maintaining competitive test performance across various kernels and datasets.
Learning the kernel parameters for Gaussian processes is often the computational bottleneck in applications such as online learning, Bayesian optimization, or active learning. Amortizing parameter inference over different datasets is a promising approach to dramatically speed up training time. However, existing methods restrict the amortized inference procedure to a fixed kernel structure. The amortization network must be redesigned manually and trained again in case a different kernel is employed, which leads to a large overhead in design time and training time. We propose amortizing kernel parameter inference over a complete kernel-structure-family rather than a fixed kernel structure. We do that via defining an amortization network over pairs of datasets and kernel structures. This enables fast kernel inference for each element in the kernel family without retraining the amortization network. As a by-product, our amortization network is able to do fast ensembling over kernel structures. In our experiments, we show drastically reduced inference time combined with competitive test performance for a large set of kernels and datasets.