Unraveling the Black Box of Neural Networks: A Dynamic Extremum Mapper
This work addresses interpretability and training issues like gradient vanishing and overfitting for neural network researchers, though it appears incremental as it builds on existing mathematical frameworks.
The authors challenge the notion of neural networks as black boxes by proving that generalization arises from dynamically mapping datasets to model function extrema, with extrema count positively correlated with parameter count, and propose a novel algorithm based on solving linear equations instead of backpropagation.
We point out that neural networks are not black boxes, and their generalization stems from the ability to dynamically map a dataset to the extrema of the model function. We further prove that the number of extrema in a neural network is positively correlated with the number of its parameters. We then propose a new algorithm that is significantly different from back-propagation algorithm, which mainly obtains the values of parameters by solving a system of linear equations. Some difficult situations, such as gradient vanishing and overfitting, can be simply explained and dealt with in this framework.