Vector Quantized Bayesian Neural Network Inference for Data Streams
This addresses the computational bottleneck for using BNNs in low-latency data stream applications, offering a practical solution for real-time uncertainty estimation.
The paper tackles the high computational cost of Bayesian neural networks (BNNs) for data streams by proposing VQ-BNN, which approximates BNN inference with a single neural network execution and temporal smoothing, achieving significantly faster performance while maintaining or improving predictive results compared to BNNs.
Bayesian neural networks (BNN) can estimate the uncertainty in predictions, as opposed to non-Bayesian neural networks (NNs). However, BNNs have been far less widely used than non-Bayesian NNs in practice since they need iterative NN executions to predict a result for one data, and it gives rise to prohibitive computational cost. This computational burden is a critical problem when processing data streams with low-latency. To address this problem, we propose a novel model VQ-BNN, which approximates BNN inference for data streams. In order to reduce the computational burden, VQ-BNN inference predicts NN only once and compensates the result with previously memorized predictions. To be specific, VQ-BNN inference for data streams is given by temporal exponential smoothing of recent predictions. The computational cost of this model is almost the same as that of non-Bayesian NNs. Experiments including semantic segmentation on real-world data show that this model performs significantly faster than BNNs while estimating predictive results comparable to or superior to the results of BNNs.