Partial Trace-Class Bayesian Neural Networks
This work addresses the scalability problem for practitioners needing reliable uncertainty quantification in deep learning, though it is incremental as it builds on existing trace-class priors.
The authors tackled the high computational cost of Bayesian neural networks (BNNs) by proposing partial trace-class BNNs, which achieve comparable uncertainty quantification with significantly fewer Bayesian parameters, leading to improved speed and memory efficiency as verified in simulations.
Bayesian neural networks (BNNs) allow rigorous uncertainty quantification in deep learning, but often come at a prohibitive computational cost. We propose three different innovative architectures of partial trace-class Bayesian neural networks (PaTraC BNNs) that enable uncertainty quantification comparable to standard BNNs but use significantly fewer Bayesian parameters. These PaTraC BNNs have computational and statistical advantages over standard Bayesian neural networks in terms of speed and memory requirements. Our proposed methodology therefore facilitates reliable, robust, and scalable uncertainty quantification in neural networks. The three architectures build on trace-class neural network priors which induce an ordering of the neural network parameters, and are thus a natural choice in our framework. In a numerical simulation study, we verify the claimed benefits, and further illustrate the performance of our proposed methodology on a real-world dataset.