On weight and variance uncertainty in neural networks for regression tasks
This work addresses uncertainty estimation in neural networks for regression tasks, but it is incremental as it builds on existing weight uncertainty methods.
The paper tackles the problem of improving prediction performance in Bayesian neural networks for regression by incorporating variance uncertainty, showing that it enhances generalization, with results demonstrated on a function approximation example and the riboflavin genetic dataset.
We consider the problem of weight uncertainty proposed by [Blundell et al. (2015). Weight uncertainty in neural network. In International conference on machine learning, 1613-1622, PMLR.] in neural networks {(NNs)} specialized for regression tasks. {We further} investigate the effect of variance uncertainty in {their model}. We show that including the variance uncertainty can improve the prediction performance of the Bayesian {NN}. Variance uncertainty enhances the generalization of the model {by} considering the posterior distribution over the variance parameter. { We examine the generalization ability of the proposed model using a function approximation} example and {further illustrate it with} the riboflavin genetic data set. {We explore fully connected dense networks and dropout NNs with} Gaussian and spike-and-slab priors, respectively, for the network weights.