Concentration inequalities and optimal number of layers for stochastic deep neural networks
This work provides theoretical guarantees for stochastic neural networks, which is incremental for researchers in machine learning theory.
The authors derived concentration inequalities for the outputs of stochastic deep neural networks (SDNNs) and introduced an expected classifier with probabilistic error bounds, applying this to a feedforward network with ReLU activation.
We state concentration inequalities for the output of the hidden layers of a stochastic deep neural network (SDNN), as well as for the output of the whole SDNN. These results allow us to introduce an expected classifier (EC), and to give probabilistic upper bound for the classification error of the EC. We also state the optimal number of layers for the SDNN via an optimal stopping procedure. We apply our analysis to a stochastic version of a feedforward neural network with ReLU activation function.