Synaptic Sampling of Neural Networks
This addresses the problem of high computational costs for uncertainty quantification in neural networks, offering a method suited for emerging probabilistic hardware, though it appears incremental as it builds on existing sampling approaches.
The paper tackles the challenge of enabling probabilistic neural networks to quantify uncertainty efficiently by introducing the scANN technique, which samples neural networks by treating weights as Bernoulli coin flips, achieving performance nearly matching deterministic methods while describing output uncertainty.
Probabilistic artificial neural networks offer intriguing prospects for enabling the uncertainty of artificial intelligence methods to be described explicitly in their function; however, the development of techniques that quantify uncertainty by well-understood methods such as Monte Carlo sampling has been limited by the high costs of stochastic sampling on deterministic computing hardware. Emerging computing systems that are amenable to hardware-level probabilistic computing, such as those that leverage stochastic devices, may make probabilistic neural networks more feasible in the not-too-distant future. This paper describes the scANN technique -- \textit{sampling (by coinflips) artificial neural networks} -- which enables neural networks to be sampled directly by treating the weights as Bernoulli coin flips. This method is natively well suited for probabilistic computing techniques that focus on tunable stochastic devices, nearly matches fully deterministic performance while also describing the uncertainty of correct and incorrect neural network outputs.