Uncertainty propagation in neural networks for sparse coding
This addresses uncertainty estimation in sparse coding for applications like signal processing, but appears incremental as it builds on existing Bayesian and sparse coding methods.
The paper tackles the problem of uncertainty propagation in neural networks for sparse coding by proposing a novel method to propagate uncertainty through the soft-thresholding nonlinearity, representing distributions as spike and slab, and designing a Bayesian neural network that uses this method and Bayesian inference.
A novel method to propagate uncertainty through the soft-thresholding nonlinearity is proposed in this paper. At every layer the current distribution of the target vector is represented as a spike and slab distribution, which represents the probabilities of each variable being zero, or Gaussian-distributed. Using the proposed method of uncertainty propagation, the gradients of the logarithms of normalisation constants are derived, that can be used to update a weight distribution. A novel Bayesian neural network for sparse coding is designed utilising both the proposed method of uncertainty propagation and Bayesian inference algorithm.