Meta-learning to Calibrate Gaussian Processes with Deep Kernels for Regression Uncertainty Estimation
This work addresses uncertainty calibration for regression in few-shot learning, which is an incremental improvement over existing meta-learning methods.
The paper tackles the problem of poor uncertainty estimation in Gaussian processes with deep kernels for regression tasks by proposing a meta-learning method that calibrates uncertainty using data from various tasks, resulting in improved uncertainty estimation while maintaining high regression performance in few-shot settings.
Although Gaussian processes (GPs) with deep kernels have been successfully used for meta-learning in regression tasks, its uncertainty estimation performance can be poor. We propose a meta-learning method for calibrating deep kernel GPs for improving regression uncertainty estimation performance with a limited number of training data. The proposed method meta-learns how to calibrate uncertainty using data from various tasks by minimizing the test expected calibration error, and uses the knowledge for unseen tasks. We design our model such that the adaptation and calibration for each task can be performed without iterative procedures, which enables effective meta-learning. In particular, a task-specific uncalibrated output distribution is modeled by a GP with a task-shared encoder network, and it is transformed to a calibrated one using a cumulative density function of a task-specific Gaussian mixture model (GMM). By integrating the GP and GMM into our neural network-based model, we can meta-learn model parameters in an end-to-end fashion. Our experiments demonstrate that the proposed method improves uncertainty estimation performance while keeping high regression performance compared with the existing methods using real-world datasets in few-shot settings.