Nonlinear Weighted Directed Acyclic Graph and A Priori Estimates for Neural Networks
This work provides theoretical insights into neural network generalization, which is an incremental contribution to the field of machine learning theory.
The authors tackled the problem of understanding the generalization power of deep neural networks by proposing a novel graph theoretical formulation for various architectures and extending a priori error estimates to DenseNet, showing that under mild conditions, estimation error bounds are independent of input dimension.
In an attempt to better understand structural benefits and generalization power of deep neural networks, we firstly present a novel graph theoretical formulation of neural network models, including fully connected, residual network (ResNet) and densely connected networks (DenseNet). Secondly, we extend the error analysis of the population risk for two layer network \cite{ew2019prioriTwo} and ResNet \cite{e2019prioriRes} to DenseNet, and show further that for neural networks satisfying certain mild conditions, similar estimates can be obtained. These estimates are a priori in nature since they depend sorely on the information prior to the training process, in particular, the bounds for the estimation errors are independent of the input dimension.