Exact Upper and Lower Bounds for the Output Distribution of Neural Networks with Random Inputs
This provides exact error guarantees for predictive output distributions in neural networks, addressing a critical need for reliability in applications like safety-critical systems, though it is incremental in extending bounds to broader network types.
The paper tackles the problem of bounding the output distribution of neural networks with stochastic inputs, deriving exact upper and lower bounds for the cumulative distribution function that converge to the true distribution as resolution increases, applicable to feedforward and convolutional networks with common activation functions.
We derive exact upper and lower bounds for the cumulative distribution function (cdf) of the output of a neural network (NN) over its entire support subject to noisy (stochastic) inputs. The upper and lower bounds converge to the true cdf over its domain as the resolution increases. Our method applies to any feedforward NN using continuous monotonic piecewise twice continuously differentiable activation functions (e.g., ReLU, tanh and softmax) and convolutional NNs, which were beyond the scope of competing approaches. The novelty and instrumental tool of our approach is to bound general NNs with ReLU NNs. The ReLU NN-based bounds are then used to derive the upper and lower bounds of the cdf of the NN output. Experiments demonstrate that our method delivers guaranteed bounds of the predictive output distribution over its support, thus providing exact error guarantees, in contrast to competing approaches.