Bayesian Image Classification with Deep Convolutional Gaussian Processes
This work addresses the need for calibrated uncertainties in image classification systems, offering a Bayesian alternative to dropout-based methods, though it appears incremental in advancing convolutional GPs.
The authors tackled the problem of improving Gaussian processes (GPs) for image classification by developing a translation-insensitive convolutional kernel and deep convolutional GPs, resulting in enhanced performance and better uncertainty estimates compared to existing methods.
In decision-making systems, it is important to have classifiers that have calibrated uncertainties, with an optimisation objective that can be used for automated model selection and training. Gaussian processes (GPs) provide uncertainty estimates and a marginal likelihood objective, but their weak inductive biases lead to inferior accuracy. This has limited their applicability in certain tasks (e.g. image classification). We propose a translation-insensitive convolutional kernel, which relaxes the translation invariance constraint imposed by previous convolutional GPs. We show how we can use the marginal likelihood to learn the degree of insensitivity. We also reformulate GP image-to-image convolutional mappings as multi-output GPs, leading to deep convolutional GPs. We show experimentally that our new kernel improves performance in both single-layer and deep models. We also demonstrate that our fully Bayesian approach improves on dropout-based Bayesian deep learning methods in terms of uncertainty and marginal likelihood estimates.