Quadratic number of nodes is sufficient to learn a dataset via gradient descent
This work provides a theoretical guarantee for efficient training in neural networks, though it is incremental as it refines existing bounds.
The authors proved that a two-layer neural network with a sufficiently large number of neurons can learn a dataset optimally via gradient descent in linear time, improving upon previous threshold bounds for this condition.
We prove that if an activation function satisfies some mild conditions and number of neurons in a two-layered fully connected neural network with this activation function is beyond a certain threshold, then gradient descent on quadratic loss function finds the optimal weights of input layer for global minima in linear time. This threshold value is an improvement over previously obtained values. We hypothesise that this bound cannot be improved by the method we are using in this work.