Robustness Guarantees for Bayesian Inference with Gaussian Processes
This work addresses safety-critical applications like robotics and control by offering formal guarantees against adversarial examples, though it is incremental as it builds on existing Gaussian process theory.
The paper tackles the problem of providing formal robustness guarantees for Bayesian inference against input perturbations, defining a measure to ensure predictions remain close within a set and deriving tight upper bounds using Gaussian processes and the Borell-TIS inequality, with evaluation on GP regression and neural networks on MNIST showing applicability.
Bayesian inference and Gaussian processes are widely used in applications ranging from robotics and control to biological systems. Many of these applications are safety-critical and require a characterization of the uncertainty associated with the learning model and formal guarantees on its predictions. In this paper we define a robustness measure for Bayesian inference against input perturbations, given by the probability that, for a test point and a compact set in the input space containing the test point, the prediction of the learning model will remain $δ-$close for all the points in the set, for $δ>0.$ Such measures can be used to provide formal guarantees for the absence of adversarial examples. By employing the theory of Gaussian processes, we derive tight upper bounds on the resulting robustness by utilising the Borell-TIS inequality, and propose algorithms for their computation. We evaluate our techniques on two examples, a GP regression problem and a fully-connected deep neural network, where we rely on weak convergence to GPs to study adversarial examples on the MNIST dataset.