Out-of-Distribution Detection from Small Training Sets using Bayesian Neural Network Classifiers
This addresses reliability and safety in AI for scenarios with small datasets, though it is incremental as it builds on existing Bayesian neural network approaches.
The paper tackled the problem of out-of-distribution detection with limited training data by introducing Bayesian posthoc scores based on expected logit vectors, showing that Bayesian methods outperform deterministic ones on MNIST and CIFAR-10 with 5000 or fewer training samples.
Out-of-Distribution (OOD) detection is critical to AI reliability and safety, yet in many practical settings, only a limited amount of training data is available. Bayesian Neural Networks (BNNs) are a promising class of model on which to base OOD detection, because they explicitly represent epistemic (i.e. model) uncertainty. In the small training data regime, BNNs are especially valuable because they can incorporate prior model information. We introduce a new family of Bayesian posthoc OOD scores based on expected logit vectors, and compare 5 Bayesian and 4 deterministic posthoc OOD scores. Experiments on MNIST and CIFAR-10 In-Distributions, with 5000 training samples or less, show that the Bayesian methods outperform corresponding deterministic methods.