Uncertainty Propagation in Convolutional Neural Networks: Technical Report
This work addresses uncertainty quantification in CNNs for researchers, but it is incremental as it builds on existing methods without introducing new paradigms.
The authors tackled the problem of propagating uncertainty through convolutional neural network components, deriving approximations for the moments of various linear and nonlinear layers and loss functions.
In this technical report we study the problem of propagation of uncertainty (in terms of variances of given uni-variate normal random variables) through typical building blocks of a Convolutional Neural Network (CNN). These include layers that perform linear operations, such as 2D convolutions, fully-connected, and average pooling layers, as well as layers that act non-linearly on their input, such as the Rectified Linear Unit (ReLU). Finally, we discuss the sigmoid function, for which we give approximations of its first- and second-order moments, as well as the binary cross-entropy loss function, for which we approximate its expected value under normal random inputs.